Database of software measurements

I discovered the existence of these two sites:

NIST National Software Reference Library (NSRL)
KnownGoods.org

This seems a lofty goal, to actually index all “legitimate” software. But I suppose capturing the common cases is straightforward enough.

Advertisements

What’s the occasion?

I posted some things today that aren’t just dry technical hoops through which I’ve recently jumped; what gives? Well, it’s May! This has always been my favorite time of year, because it’s getting nice outside, the school year is ending (not so true for me anymore…), and I usually see a big surge in the amount of available free time I have. In the past, this has resulted in some excellent trips such as biking from Pittsburgh to Philly, road-tripping to Moab for some biking, or road-tripping to an internship on the west coast (which involved many stops for some biking). I just returned from rowing in this year’s Dad Vail Regatta, although I don’t count that as a full-on vacation, since it only lasts 2 days. Sadly, I am not a national champion.

As far as trips go, I will be going to Oakland, CA for the IEEE Symposium on Security and Privacy. While I don’t have a paper there this year (I did have two last year…), some other folks have extended one of my papers from last year and I look forward to seeing their presentation of that work.

Shortly thereafter I will be going to Boston, MA for the USENIX Annual Technical Conference, where I will be presenting a new paper.

After that, it’s wedding season, as all my friends are getting married.

Micro- versus monolithic kernels

Some significant figures (and plenty of insignificant ones) are fighting about micro- versus monolithic- kernels again, with microkernels making a very strong showing on the grounds of security and reliability. The issue is getting a lot of attention right now, not that anything technical has magically changed. I find this exciting, as this last happened in 1992 when I was 12 years old, and I’m tired of people talking about uKernels like everything interesting to be said about them is already out there. They seem like a good idea to me!

IEEE ran the article “Can We Make Operating Systems Reliable and Secure?”

Linus wrote something.

Tanenbaum wrote about it.

Shapiro wrote about it.

And so have a lot of other people.

And shit, so did I. 😉

Walking to the Sky and the Gates Center

Walking to the Sky has been erected on CMU’s campus! Here is a link to the artist’s website, and here is a link to a CMU Press Release. This project is not very popular among the CMU students because most people seem to think it’s ugly (I think it’s wierd enough that its being ugly is acceptable). I first learned about the plan to put it up on campus from misc.market, a bboard at CMU which is designed for selling things (and does this well) but exists in an ever-changing state of unmoderated bliss (I base my statement that most CMU students think the sculpture is ugly on the assumption that people who flame each other on misc.market compose a representative subset of the CMU student population). My main concern, inspired by a misinformed misc.market poster, was that the sculpture was going to ruin the ability to, e.g., throw a frisbee on The Cut. The sculpture is in one corner of The Cut, however, so I don’t think it hurts anything. It’s so bizarre that I actually think its presence will be fun. Everybody who comes to campus will say, “what the hell is that?” and I’ll have a story to tell them.

The Gates Center has also started to become reality. After many weeks of putting up fences and evicting people, some stuff actually got demolished today. Stuff in this instance is the garages between Newell-Simon and Hamburg Halls. I would say the old student center’s days are numbered, also.

SVN and commit emails

First of all, almost all of the problems I’ve had with SVN have been caused by the scripts used to send commit emails. My favorite problem is when you commit a lot at once and the script simply dies because various parts of the email infrastructure aren’t equipped to deal with “a lot” of message length (i.e., 10s of megabytes or more).

Version 2.0 of the commitmessage python scripts has an option called summaryThreshold which can cope with this. I suggest you install and use it, which I just did.

Now, during the installation process, I had a hickup. I tried to test the new script as follows:

sudo -u www-data ./post-commit /var/lib/svn/REPO 1

and it worked. Great! Then I tried to commit something new, and it hung for a long long time before eventually causing the SVN client to core dump (very classy; my other big complaint about SVN is that the client will often dump core and terrify me when the server is actually doing just fine). I have WebSVN configured for my REPO, so I went to have a look. To my surprise, the “author” category was blank for some of my commits, including the most recent ones. When I tried to invoke the post-commit script manually, it would only succeed if given a revision number for which there is an author.

The SVN FAQ told me about situations where there is no author for a commit. This happens when anonymous commits are allowed, since it’s impossible to tell who the author is. It turns out I had anonymous commits enabled, as I was working with a toy repository. So, if *=rw exists in the access file, meaning anybody can read or write to the repository, then SVN does not keep track of an author. The commit script cannot cope with this. I have no solution other than to enable access control, which is what I really want anyways, so no big deal.

WordPress themes

Themes are really simple to add. Here is a list of lots of themes. Pick one, download the .zip file, unzip it to the right place, and you’re done. The black and red one I just activated is called “anarchy”. I picked it because it uses the full width of the user’s browser. Source code is hard enough to read without unnecessary line wraps.

backupninja is tricky

I am using backupninja (Debian package) to backup parts of the data on a server to another machine. The server has home directories, web stuff, and an SVN repository. Backupninja seemed a good choice because it is very easy to tell it to export the SVN repository to a flat file before executing the backup.

I copied two files from /usr/share/backup/backupninja/examples to /etc/backup.d:
example.svn -> 10.subversion.svn
example.rdiff -> 90.rdiff

The defaults in the SVN file worked fine for me. I edited 90.rdiff to include the data I want backed up.

Recently, however, a network glitch caused the rdiff backup to fail with the following output (which I received in my usual status email from backupninja):


== warnings from /etc/backup.d/90.rdiff ==

Warning: Exception '' raised of class 'exceptions.AssertionError': File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 295, in error_check_Main try: Main(arglist) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 315, in Main take_action(rps) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 271, in take_action elif action == "backup": Backup(rps[0], rps[1]) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 328, in Backup backup_final_init(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 425, in backup_final_init checkdest_if_necessary(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 820, in checkdest_if_necessary need_check = checkdest_need_check(dest_rp) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 810, in checkdest_need_check if not force: curmir_incs[0].conn.regress.check_pids(curmir_incs) File "/usr/lib/python2.3/site-packages/rdiff_ba
ckup/connection.py", line 448, in __call__ return apply(self.connection.reval, (self.name,) + args) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 367, in reval for arg in args: self._put(arg, req_num) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 139, in _put else: self._putobj(obj, req_num) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 144, in _putobj self._write("o", pickle.dumps(obj, 1), req_num) File "/usr/lib/python2.3/pickle.py", line 1386, in dumps Pickler(file, protocol, bin).dump(obj) File "/usr/lib/python2.3/pickle.py", line 231, in dump self.save(obj) File "/usr/lib/python2.3/pickle.py", line 293, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.3/pickle.py", line 614, in save_list self._batch_appends(iter(obj)) File "/usr/lib/python2.3/pickle.py", line 647, in _batch_appends save(x) File "/usr/lib/python2.3/pickle.py", line 293, in save f(self, obj
) # Call unbound method with explicit self File "/usr/lib/python2.3/pickle.py", line 737, in save_inst stuff = getstate() File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 754, in __getstate__ assert self.conn is Globals.local_connection Traceback (most recent call last): File "/usr/bin/rdiff-backup", line 23, in ? rdiff_backup.Main.error_check_Main(sys.argv[1:]) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 295, in error_check_Main try: Main(arglist) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 315, in Main take_action(rps) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 271, in take_action elif action == "backup": Backup(rps[0], rps[1]) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 328, in Backup backup_final_init(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 425, in backup_final_init checkdest_if_necessary(rpout) File "/usr/lib/python2.3/site-pa
ckages/rdiff_backup/Main.py", line 820, in checkdest_if_necessary need_check = checkdest_need_check(dest_rp) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 810, in checkdest_need_check if not force: curmir_incs[0].conn.regress.check_pids(curmir_incs) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 448, in __call__ return apply(self.connection.reval, (self.name,) + args) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 367, in reval for arg in args: self._put(arg, req_num) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 139, in _put else: self._putobj(obj, req_num) File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 144, in _putobj self._write("o", pickle.dumps(obj, 1), req_num) File "/usr/lib/python2.3/pickle.py", line 1386, in dumps Pickler(file, protocol, bin).dump(obj) File "/usr/lib/python2.3/pickle.py", line 231, in dump self.save(obj) File "/usr/lib/python
2.3/pickle.py", line 293, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.3/pickle.py", line 614, in save_list self._batch_appends(iter(obj)) File "/usr/lib/python2.3/pickle.py", line 647, in _batch_appends save(x) File "/usr/lib/python2.3/pickle.py", line 293, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.3/pickle.py", line 737, in save_inst stuff = getstate() File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 754, in __getstate__ assert self.conn is Globals.local_connection AssertionError Fatal Error: Lost connection to the remote system
Warning: Failed backup up source

This confused me badly and I ended up trying a bunch of things which did not work. In the end, I moved my current backups out of the way on the backup machine and started anew.

Error messages I received in the mean time include:


== warnings from /etc/backup.d/90.rdiff ==

Warning: Previous backup seems to have failed, regressing destination now. Warning, could not find mirror_metadata file. Metadata will be read from filesystem instead. Fatal Error: No metadata for time Thu Apr 27 18:38:23 2006 (1146177503) found, cannot regress Fatal Error: Lost connection to the remote system
Warning: Failed backup up source

as well as:


== warnings from /etc/backup.d/90.rdiff ==

Warning: Exception 'Found too many current_mirror incs!' raised of class 'exceptions.AssertionError': File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 295, in error_check_Main try: Main(arglist) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 315, in Main take_action(rps) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 271, in take_action elif action == "backup": Backup(rps[0], rps[1]) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 328, in Backup backup_final_init(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 425, in backup_final_init checkdest_if_necessary(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 820, in checkdest_if_necessary need_check = checkdest_need_check(dest_rp) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 811, in checkdest_need_check assert len(curmir_incs) == 2, "Found too many current_mirror incs!" Traceback
(most recent call last): File "/usr/bin/rdiff-backup", line 23, in ? rdiff_backup.Main.error_check_Main(sys.argv[1:]) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 295, in error_check_Main try: Main(arglist) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 315, in Main take_action(rps) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 271, in take_action elif action == "backup": Backup(rps[0], rps[1]) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 328, in Backup backup_final_init(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 425, in backup_final_init checkdest_if_necessary(rpout) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 820, in checkdest_if_necessary need_check = checkdest_need_check(dest_rp) File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 811, in checkdest_need_check assert len(curmir_incs) == 2, "Found too many current_mirror
incs!" AssertionError: Found too many current_mirror incs! Fatal Error: Lost connection to the remote system
Warning: Failed backup up source

Here is an rdiff backup wiki which provided some of the suggestions which I tried unsuccessfully. Also, here is the rdiff-backup-users mailing list archive, which could prove to be useful.

My advice is this: if rdiff-backup fails, add in the –force option and retry. Hopefully that alone is sufficient to fix it. If that doesn’t fix it, I don’t know what to do.