bfields's blog

Last week was the CITI-hosted NFSv4 "bakeathon", so was hectic for me. In particular, I'd promised a brief presentation on Wednesday, and had very little time to prepare; so I gathered my small amount of data for the presentation last Sunday, then put together the presentation Tuesday and Wednesday--I was still writing slides as Andy was driving me to the meeting.

Monday and Wednesday night both I went to the Arbor Brewing Company with bakeathon people. Thursday I went to the Corner Brewery to see Dave's brother Robin play with Jenna Mammina. The music was fun. I think Robin is an amazing accompanist. It was also good to see Jenna's brother Nino there singing--last time we saw him he was in a wheelchair and not talking.

People from work showed up a little later. Between Dave's family, and Jenna's family, and work people, and Murph, who was chatting with people at a nearby table, it was sort of fun to be surrounded by people I knew for different reasons.

The Corner Brewery has beer but only allows ordering out for food, so after a beer I left with Trond and company for the Sidetrack. Dave and Robin joined us a little later. It was all very pleasant. But Friday I was too tired to do much.

I stopped by juggling Saturday but wasn't too motivated, so went back to work for a while, then went to a birthday party. I still was feeling a little low on energy, though, and didn't talk to people much.

Sunday I worked some more at the Media Union.

I've been listening to Miles Davis's version of "Someday My Prince Will Come". My favorite moment is the start of the first trumpet solo, where he plays three eighth notes that kick off the chorus and temporarily give it a really distinct rhythm.

I also hadn't noticed before how they add an extra 8 bar pedal point before and after Coltrane's solo--it's as if they're saying "watch out! Here he comes!"

I have too many CD's; any one of them could last me a year if I listened hard enough.

I recently read the 1985 paper "Design and Implementation or the Sun Network Filesystem", but didn't find much in there that was interesting to me.

Sunday, in a quest to figure out how chmod is actually used out in the wild, I went through log output we've been getting from a kernel patch of Andreas's that logs uses of the chmod system call. I also grepped through the entire fieldses.org filesystem looking for scripts that call "chmod".

Why are there so many programs that create a file and then immediately chmod it? Why couldn't they just get the umask and the create mode right?

Today was the first day of the Bakeathon here in Ann Arbor. I did a little more ACL work, some debugging, tried to help people set up a little, and talked with Greg Banks a little.

Several of us went out to the Arbor Brewing Company afterwards. It was pleasant enough, though a little noisy.

I was rolling down East Medical Center Drive at a pretty good clip this afternoon when my steering suddenly went a little funny and I heard a little "wump" and some metallic klangs.

Looking back confirmed my suspicions: the pannier hung on the left side of my rear wheel had thrown itself off onto the street.

Fortunately, the closest car was far enough back to have plenty of time to drive around my stuff. And all that had actually scattered out of the pannier was my keys and the two halves of my bicycle lock.

This isn't the first time something like this has happened, though it's probably the worst. Some day I'm afraid I'm going to lose control of the bike, or my laptop is going to slide into the next lane and get run over.

So I need to figure out some better way to carry stuff. Or maybe these panniers just need some more adjustment.

Kleiman's "Vnodes: An Architecture for Multiple File System Types in Sun UNIX" describes a basic architecture that's pretty familiar at this point.

One design goal I wouldn't have thought of:

All file system operations should be atomic. In other words, the set of interface operations should be at a high enough level so that there is no need for locking (hard locking, not user advisory locking) across several operations. Locking, if required, should be left up to the file system implementation dependent layer. For example, if a relatively slow computer running a remote file system requires a supercomputer server to lock a file while it does several operations, the users of the supercomputer would be noticeably affected. It is much better to give the file system dependent code full information about what operation is being done and let it decide what locking is necessary and practical.

Linux's VFS doesn't really work that way--the VFS operations are too fine-grained. But that does mean we end up having to do special things for distributed filesystems (e.g. the intents stuff).

Note that VFS described in this paper has no common inode lock, for example; that's left up to the filesystem.

Monday night we want with Paul, Dave, and Bill C. to the Saline Community Fair for the demolition derby. Well, it seemed like one of those things that's worth trying once.

The basic idea is really simple: 10 cars, give or take, all start at once. The last car still moving wins. Referees may disqualify cars or stop the whole thing if (as happens once or twice) it looks like an engine fire might get out of control. Then I guess there's a whole bunch of rules about how the cars are prepared. The one obvious rule was that the doors all had to be welded shut.

I think there may have been five rounds altogether, the fifth featuring winners (well, survivors anyway) of the previous four. That tried my patience a bit. But it was still pretty funny.

Paul described it as like an action movie but real, and with all the in-between non-action parts removed. There was lots of noise and confusion. It's suprising how completely the body of a car can be mutilated without stopping it from running. Even the occasional missing wheel or shredded tire didn't always stop them.

So while I don't think I need to see another, I definitely don't regret going.

A few years ago the Michigan Daily's Crime Notes section included an item under the headline "Boyfriend reports missing girlfriend, later found":

A male resident a Cozen's Residence Hall became worried Saturday evening after he was unable to locate his girlfriend, DPS reports state. The man had previously made plans with his girlfriend, who was later located.

You read the article, and you think, that could not possibly be improved on--love lost and regained all in 35 words. And then you realize the headline does it all in 6 words. It's a masterpiece.

I just have the clipping now, stuck to my office door, and don't even know what year it came from.

Josh, Valree, and two of their friends are all leaving for Los Angeles, so they had a party at their friends' house in Ypsi on Saturday night. We didn't know many of the people there, but they had both a fire-eater and a band, which had been formed just days before for the party and had only 3 songs, the first of which was "paper, scissors,... rock!", also the name of the band.

So you can't complain about a party like that.

The other night Sara and I had one of these instant indian meals that come in a box containing a foil patch with your channa masala or palak paneer ready to be heated.

This time we were surprised to find a CD stuck to the foil patch, with a note to buy more of their meals and collect all five volumes. So of course we put it on and listened to it with dinner. And actually it was very good--it was a few classical indian pieces, a genre of music I like but know almost nothing about.

The standard for supermarket convenience food has now been raised--I'll always expect dinner music packaged with my dinner from now on.

My day at work alternated between trying to put out a new release of my kernel patches (complicated by a failing machine at citi making some of our services unreliable), working on various ACL-related stuff (mainly talking over things with Andreas and then trying to read through some of his code), and helping Fred with some debugging (a little; Fred was the one that actually figured it out).

Some days a problem really grabs me and I just can't stop thinking about it; this wasn't such a day.

After work I met Trond, Laura, Sara, and (eventually) Paul at Dominicks. Trond suggested the nfsd readahead problem as the sort of thing that should keep a person up at night.

In particular, the problem is this: every time an application asks for data from a file, you have to go read the data from disk. But that's terribly inefficient. It takes eons (ok, milliseconds--but that's eons on today's hardware, where a processor executes an instruction every nanosecond) to move the disk head to the right place, wait for the disk to spin around to the right spot, and read the data. So, ideally, you'd like the data to already have been read. How is that possible? Well, it's not always possible for the operating system to predict what's going to be asked for next. But often it is, because often applications just read through whole files sequentially from beginning to end.

So any modern operating system recognizes when an application is reading straight through a file from beginning to end, and starts anticipating by performing "readahead"--reading the next few chunks of the file before they're requested, assuming that they'll be needed soon.

The problem comes when you throw NFS into the mix--now the application is split from the disk by a network, and the read requests may sometimes be switched around and arrive in a different order. So even though the application is reading through the file in order, it looks to the NFS server like they're going back and forth a little, and the standard readahead algorithm fails because it doesn't recognize that this is basically not that different from sequential reads.

I actually read a paper recently where they dealt with this problem, but can't for the life of me remember where it was....

When I got home I tried reading through Pike and Weinberger's "the Hideous Name", but found it kind of a pointless paper.

McKusick and Kowalski's "Fsck -- The UNIX File System Check Program" (1985) rehashes some details from the FFS paper before talking about corruption.

Note that all writes required to deallocate a block from an inode are done synchronously, and all directory operations are done synchronously, to prevent, for example, a block ever being referenced from two different inodes (no longer an automatically recoverable situation since you can't decide which it should belong to).

The rest of the paper is a list of the various sanity checks performed by fsck; mostly fairly obvious. There's no discussion of performance, which must be terrible--multiple passes through the whole filesystem seem to be required, and memory requirements for some of the calculations are probably unbounded.