RSS LJ

May 30, 2004

The post of EVIL ()

by fluffy at 10:43 PM
Archive ID 666 needs a suitably evil post to fill in its space.

So, how about this idea for remote file search?

Okay, one of the really cool things about HFS+ on MacOS X is that it builds a metadata index at the filesystem level. It's like the standard UNIX locate command, except it's updated with every filesystem op, so searches are always up-to-the-second.

However, AFP (the HFS+ equivalent of SMB, which is basically HFS+ tunnelled over an ssh connection — or, looking the other way around, basically like SMB only encrypted and with UNIX permission and HFS+ resource fork support) doesn't seem to support this index. When you do a remote search on an AFP share, it seems that the Finder actually has to fetch the entire directory tree and all of its metadata and then do the search locally.

So why not support the index search at the AFP level?

Seriously, Apple should just add a 'find' primitive to the AFP protocol, which could then have the server on the other end use its own nicely-maintained metadata index to do the search, as a scatter-gather operation.

it'd make finding files on a cluster of computers really easy, and do even more to help with Apple's "zero-configuration grid of computing resources" push.

Are you listening, Apple? You guys really need to hire me. Stop ignoring my résumé! Seriously!

Please?!

(Incidentally, in the early days, Jobs and Wozniak tried to sneak the number 666 in wherever they could. Like, the original price of the Apple II was $666.66. So the timing of this idea was oddly fitting.)

Of course, this idea doesn't have to happen on HFS+ or AFP. Any filesystem-level protocol could be extended to support remote scatter-gather operations. Even if the remote system just runs find -name it'd be way better than the local system having to grab all of the remote metadata just to perform a local query! Why grab 50MB of data to find a single 100-byte entry when you could just send a 20-byte query and receive a 100-byte response? I'd also think that the server network load incurred by a remote find query would be way less than the srver load incurred by a remote filesystem traversal... The worst-case is a search which matches everything, which is essentially what has to happen all the time anyway.

Comments

#2589 05/30/2004 10:41 pm You call that evil?
That's not an evil mechanism for file search. I have one word for you:

gnutellafs

(scroll down to section 2.5 of that page)

Basically, you mount the gnutellafs filesystem somewhere. To make a search, you create a subdirectory under SEARCH with the names of your search terms. Then the results of your search appear as ordinary files within that directory.

That's evil file search.
#2590 05/30/2004 10:47 pm
Whoa.
#2591 Ali (unregistered) 05/31/2004 06:56 am Paul wins.
Okay that was so evil, it probably has burly henchprocesses, an army of inodes with faceless helmets, and a plan to take over the Internet.

o O
#2593 05/31/2004 08:15 am
'Course, gnutellafs doesn't help you to find that old email conversation you saved somewhere on some computer three years ago.