Type: atom
Encoding: UTF-8
Title: ongoing
Link: 
Description:
Creator: 

****************************************
Item: Cottage Life I: Logs
<CL-Logs#comments>
Date: 2008-06-16 07:00:17 UTC
I think that life in general and this space in particular would benefit 
from more of an outdoor flavor; words and pictures rooted in Nature. Our 
recent acquisition of a piece of Keats Island[1], should make this 
easier. Welcome to Cottage Life[2]. Any piece of Pacific Northwest 
waterfront is going to include a lot of logs.

A few trees naturally fall into the ocean when they die, but most of the 
logs that drift up on our beach represent little errors and omissions in 
the logging industry. Time was, you could make a living scooping these 
up and selling them back to the foresters; there was even a TV series 
about it[3].[A]

I believe that the rock holding the log up represents the extreme 
northernmost point of Keats Island.

Here’s a close-up of another; they become more visually interesting as 
they spend time alternating between salt water and rocky beaches. I was 
walking along this one and was struck by the extreme whiteness of the 
exposed wood. I wonder how long it took?[B]

Photo-geek note: we were on the island for five hours Saturday and I 
never felt even the slightest temptation to take the wide-angle[4] prime 
lens off the camera.

[1] /ongoing/When/200x/2008/06/11/Keats-Island
[2] /ongoing/What/The%20World/Cottage%20Life/
[3] http://en.wikipedia.org/wiki/The_Beachcombers
[4] /ongoing/When/200x/2007/05/24/Pentax-SMC-DA-21mm

[A] PS081037.png
[B] PS081035.png

Feed: ongoing
Filed under: The World/Cottage Life, The World, Cottage Life, Arts/Photos/Cameras, Arts, Photos, Cameras, The World/Places/Canada/Howe Sound, Places, Canada, Howe Sound
****************************************
Item: Deletionist Morons
<Deletionist-Morons#comments>
Date: 2008-06-15 21:34:36 UTC
For a nice safe NPOV (“Neutral Point of View”) discussion of the issues, 
see Deletionism and inclusionism in Wikipedia[1]. My experience, which 
isn’t NPOV at all, is that Deletionists are knuckle-dragging droolers, 
walking vacant spaces, and as a side-effect generally, well, what’s the 
word I’m looking for? “Wrong.”

What got me involved was word that the Deletionist undead were shambling 
in the direction of _why’s entry[2]. Oh, and by the way, apparently it’s 
somehow uncool[3] that I entered the debate[4] because I heard about it 
somewhere.

This one is obvious; _why[5] is one of the handful of people who 
constitute the public face of Ruby. Some of his published code snippets 
make shivers run up and down my spine. Several of his libraries—most 
notably, I suppose, Hpricot[6]—are very widely used. He’s written a 
popular online book[7]. I’ve never met him but would very much like to.

Yes, he’s eccentric. Yes, he conceals his real name. Yes, he would 
almost certainly be happier if the entry were removed. So what? 
Wikipedia without that entry would be less accurate, less complete, and 
less useful.

In this particular case, the arguments from the deletionists are 
jargon-laden (hint: *real* experts use language that the people they’re 
talking to can understand), and either stupid or vacuous. Which is 
entirely consistent with my experience.

Are the deletionists simply, as it seems, reveling in their own 
inadequacies? This came up on Twitter last night and Jeff Atwood put it 
well[8]: “The fatal flaw of deletionism is the mindset of deciding what 
someone else *should* find interesting.” Jeepers, what kind of 
impoverished soul does it take to delight in removing accurate and 
potentially-useful information from the permanent record based on the 
subjective and silly notion of “Notability”?

A little thought-experiment is in order: What harm would ensue were 
Wikipedia to contain an accurate if slightly boring entry on someone who 
was just an ordinary person and entirely fame-free? Well, Wikipedia’s 
“encyclopedia-ness” might be impaired... but I thought the purpose of 
Wikipedia was to serve the Net’s users, not worry about how closely it 
adheres to the traditional frameworks of the reference publishing 
industry?

I suggest the deletionist wankers go and cluster around an alternate 
online reference tome which has articles only about God, Immanuel Kant, 
and Britney Spears. Notability is not in question, so they should be 
happy.

[1] http://en.wikipedia.org/wiki/Deletionism
[2] http://en.wikipedia.org/wiki/Why_the_lucky_stiff
[3] http://en.wikipedia.org/wiki/Wikipedia:Canvassing
[4] 
http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Why_the_lucky_stiff_%282nd_nomination%29
[5] http://whytheluckystiff.net/
[6] http://code.whytheluckystiff.net/hpricot/
[7] http://poignantguide.net/ruby/
[8] http://twitter.com/codinghorror/statuses/835112269

Feed: ongoing
Filed under: Technology/Publishing/Reference, Technology, Publishing, Reference, The World/Life Online, The World, Life Online
****************************************
Item: Not Much
<Not-Much#comments>
Date: 2008-06-13 07:32:18 UTC
Two photos of not much in particular, but with explanations.

Explanation: Girls and Trucks[1].[A]

This is another part of Main Street, which is partly explained in the 
first paragraph of Main Art[2].[B]

I tried this last one in black and white, and it looked sort of stark 
and strong and lovely, but then I turned the colour back on and liked it 
better. I guess I’ll never be a Real Photographer.

[1] http://www.laurenwood.org/anyway/2008/06/09/girls-and-trucks/
[2] /ongoing/When/200x/2008/06/11/On-Main

[A] PS080961.png
[B] R0010529.png

Feed: ongoing
Filed under: Arts/Photos, Arts, Photos, The World/Places/Vancouver, The World, Places, Vancouver
****************************************
Item: Main Art
<On-Main#comments>
Date: 2008-06-12 00:19:10 UTC
Our house is near, and my office on, Main Street[1], which despite its 
name is not, nor has it ever been, the main street of anything. These 
days “The Main” is starting to be hip and fashionable, but the process 
is hampered by the fact that the buildings are, on average, old, and 
mostly weren’t very good when they were built. So it was kind of nice 
when some street art popped up on one that I walk by every day.[A][B]

It’s nice that somebody goes to the effort to do this when the results 
are so ephemeral. Whoever you are, thank you.

[1] http://en.wikipedia.org/wiki/Main_Street_%28Vancouver%29

[A] R0010514.png
[B] R0010515.png

Feed: ongoing
Filed under: Arts/Photos, Arts, Photos, Arts/Visual, Visual
****************************************
Item: Cottagers!
<Keats-Island#comments>
Date: 2008-06-11 07:39:53 UTC
Now we’re real Canadians. As of June 11th, Lauren and I own a cottage on 
Keats Island (Wikipedia[1], map[2], keatsisland.net[3]). The 
consequences include a sudden interest in remote-area Internet options.

For context on the neighborhood, Canadian cottage culture, and pretty 
pix, check out Howe Sound day[4], which documents a trip we took mostly 
to look at the property. That was almost six months ago; time moves 
slower in cottage country. The two pictures in that piece identified as 
being from the north shore of Keats are taken from the cottage. Here’s 
another.[A]

Why?

When we looked at the place we liked the view, we liked the general 
setup, we liked the accessibility from Vancouver, the price seemed OK, 
and there was nothing obviously wrong.

While we were over there some of the party had a nap and I went for a 
walk in the woods with our son, now eight.

You have to understand that Keats is not a very big island, that the 
cottages are lined up along the oceanfront and that behind them the 
ground rises steeply into really rough bush. There were some rough paths 
through the bush behind the cottages so I said to the boy “let’s check 
’em out.” We got maybe a hundred yards down the trail and he was saying 
“Dad, should we be here? Is it safe? Will we get lost? I think we should 
go back.” I realized that we were raising a 100% city boy who had 
probably never been in the woods, out of sight of human habitation, in 
his life. Suddenly the purchase seemed more attractive.

Details

It’s pretty near the northernmost point of the island. It has an 
address—several of them in fact—but the streets named are purely 
hypothetical entries on some decades-old plan and were never built. If 
you zoom in close enough in Google maps you can actually see the 
place.[B]

It’s got electricity and phone, and a very decent well shared with the 
other cottagers on that bit of shoreline. It has 150 feet of 
Pacific-ocean waterfront with a “foreshore lease” and a floating dock in 
the summer months, which is good because you can’t really drive to it. 
The house is probably 75 vertical feet above the water on a rocky cliff, 
so erosion worry is minimized and the view is good. There’s a weekend 
ferry that will drop you off at your own dock. There are two bedrooms 
and reasonably modern appliances. Plumbing is a problem.

Internet!?!?

Well, there’d be dial-up access if that were any use. In the summer, I 
look forward to the occasional weekend completely off the grid, but 
that’s not very realistic if we wanted to, now and then, spend a week or 
more. Suffice it to say that the telco is not interested in extending 
DSL facilities to anywhere this remote.

Hmm... there are two companies competing to offer satellite Internet, 
but the data rates are pretty ordinary and it’s hundreds and hundreds of 
dollars to get installed and set up. Also, we’re at the north end of the 
island and there’s a small mountain in the middle, so I bet we can’t see 
the satellite.

We have direct line-of-sight to the “Hope Point Repeater” of the Gambier 
Radio Internet Project[5] and its principal responds to email, so that 
might work.

We also have line-of-sight to most of Gibsons[6] which means that if we 
could find someone there with broadband and an open mind, we could set 
up a long-distance WiFi repeater. Does anyone reading this live in 
Gibsons?

Why Again

We tell ourselves that it’s a decent investment in times of financial 
turmoil, and that it’s for the kids, and those things are mostly true, 
but having a place where you can get up early and walk to say good 
morning to the ocean before you say anything to anyone else, that’s why 
really.

[1] http://en.wikipedia.org/wiki/Keats_Island%2C_British_Columbia
[2] 
http://maps.google.com/?ll=49.403154,-123.461266&amp;spn=0.120418,0.221443&amp;z=12
[3] http://www.keatsisland.net/
[4] /ongoing/When/200x/2008/01/22/Howe-Sound
[5] http://www.gambierrip.ca/
[6] http://en.wikipedia.org/wiki/Gibsons

[A] IMGP7549.png
[B] IMGP7517.png

Feed: ongoing
Filed under: Arts/Photos, Arts, Photos, The World/Places/Canada/Howe Sound, The World, Places, Canada, Howe Sound
****************************************
Item: RX and 1.9 and Pain
<RX-Work#comments>
Date: 2008-06-11 06:29:20 UTC
This fragment is mostly a note to myself and placeholder and might prove 
useful to someone slashing through the XML undergrowth with 
bleeding-edge Ruby. Briefly: I revived my “RX” Ruby tokenizer (see 
here[1], here[2], and here[3]) to contribute to Antonio Cangiano’s 
proposed Ruby benchmark suite[4], which I think is a Really Good Idea. I 
had a bit of pain getting the code to run on both Ruby 1.8 and 1.9, and 
then when I tried sanity-checking the output by comparing it to REXML on 
1.9, REXML blew chunks. There are, apparently, *issues* about REXML and 
1.9. Read on for details in the unlikely event that you care about any 
of this.

Benchmarking

There’s this problem in that there are a lot of plausible-looking Ruby 
implementations now (MRI, YARV, JRuby, Rubinius, IronRuby, MagLev) and 
it would be nice to compare performance. I was talking to some of the 
implementers about this and someone (Charles Nutter I think) said 
“Problem is, there’s this huge gap between running fib() and running 
Rails.” So, for example, how do we find out how fast MagLev will run 
Rails, without going through all the pain of making MagLev run Rails?

Antonio Cangiano sensibly proposed Let’s create a Ruby Benchmark 
Suite[4]; when Avi Bryant told me he’d tried my RX code on MagLev, it 
occurred to me that it might be an interesting benchmark.

RX refresher: It’s a pure automaton-based XML tokenizer whose 
performance is *totally* dependent on the efficiency of dereferencing 
integer arrays, and it turns out that mainstream Ruby really sucks at 
this.

To make it a little more competitive with REXML, the de-facto standard 
Ruby parser, I had kludged it all over the place with regex 
preprocessing to cut down on the array traffic.

So I asked Antonio whether, if I de-optimized RX to make it a pure array 
benchmark, would it be interesting for his suite, and he said yes, so I 
did.

1.8.6 vs. 1.9

Perhaps the single most visible difference between today’s Ruby and 
tomorrow’s is in the low-level string-handling API. Well, an XML parser 
*lives* entirely right there, so boy did I ever learn all about it. I 
had previously converted RX to run on 2006-vintage YARV, but I wanted 
one version of the code that would run in both 1.8.6 and 1.9. Sigh.

Here’s one of the detail issues, to give a feeling for the problems. 
Suppose you know that your input stream is in UTF-8 and you’ve read a 
buffer-full of data and you want to turn it into Unicode integer 
characters for the parser. The problem is that the buffer might end in 
the middle of a multi-byte UTF-8 character. Easy enough, a glance at the 
last byte will diagnose that. The problem is, how do you pull out the 
unsigned-integer value of the last byte of a buffer, without processing 
through the whole (potentially large) buffer, with code that runs in 
both Rubies?

I poked around on IRC and Eric Hodel[5] managed to improve on my 
original suggestion. Read it and weep:

def byte_at(s, i)
s[i, 1].unpack('C')[0]
end

REXML Ouch

RX has a primitive unit-test suite; what I do to sanity-check it at a 
high level is feed a nontrivial XML doc to it and REXML and check that 
they find the same number of elements, PIs, paragraphs, img elements 
with a src= attribute whose value ends in .jpg, and occurrences of the 
word “the” in running text.

Well, when I finally got it running in Ruby 1.9, and started the sanity 
check, REXML blew up on my document, 2.8 Meg of the input to this blog.

With a bit of poking around, I ascertained that:

-

REXML blew up differently depending whether I used its stream or DOM 
mode.
-

In the stream mode, it apparently fell over while trying to handle an 
instance of &#xbd;, but I couldn’t replicate the failure with a small 
file containing that.
-

In the DOM mode, it incorrectly reported an error on a CDATA section 
containing an XML declaration and DOCTYPE, but I couldn’t replicate the 
failure with a small file containing that.
-

I asked Sam Ruby whether I should be surprised, and a glance at his 
continuous-integration setup for 1.9[6] reveals that I shouldn’t, there 
are lots of tests failing.
-

There is quite a bit of disgruntlement about XML and Ruby right at this 
point in time; see The Status of Ruby’s libxml[7] and My Frustrations 
with REXML: Ruby’s Standard Library for Reading/Writing XML; or, Ruby’s 
Problem Is Its Type System, and Don’t Try to Tell Me Otherwise[8] and 
Does Ruby's support for XML suck? I need your input[9].
-

The issue is under discussion on the ruby-core mailing list.

Well, there you go. By the way, in Ruby 1.9’s favor, it runs the 
(simplified de-optimized) RX about three times as fast as 1.8.6. Any 
other implementors want a whack at it?

[1] /ongoing/When/200x/2006/11/09/Optimizing-Ruby
[2] /ongoing/When/200x/2006/11/15/RS-Redux
[3] /ongoing/When/200x/2006/11/23/RX-plus-YARV
[4] 
http://antoniocangiano.com/2008/06/01/help-me-create-the-ruby-benchmark-suite/
[5] http://segment7.net/
[6] http://www.intertwingly.net/blog/2008/01/31/Ruby-CI-Efforts
[7] 
http://enfranchisedmind.com/blog/2008/05/27/the-status-of-rubys-libxml/
[8] 
http://enfranchisedmind.com/blog/2008/03/24/rexml-dynamic-typing-lose/
[9] http://www.rubyflow.com/items/388

Feed: ongoing
Filed under: Technology/XML, Technology, XML, Technology/Ruby, Ruby
****************************************
Item: Flamenco Sketches
<Flamenco-Sketches#comments>
Date: 2008-06-09 06:13:33 UTC
Photographic sketches, I mean. Earlier this month, Eve[1] and Eli were 
up from Seattle to visit and after dinner, we all dropped by the Kino 
Café[2] to take in the flamenco. It was passionate and sexy and fun.

There was almost no stage lighting, and what there was in vivid reds and 
yellows; I pushed the Pentax[3]/Sigma[4] combo to the max, but it was 
tough.

There were four performers. The woman on the left was the flashier 
dancer, the white-haired fellow was a fine singer and still had a couple 
of moves left, and the guitar sounded as big as an orchestra.[A][B]

But it was the other woman, something about her face, I just couldn’t 
take my eyes off it, beautiful yes, but so alive and engaged and in the 
moment.

The black-and white was an irritated reaction at my inability to 
colour-correct the seriously-unexposed picture, but then I started to 
like it. There are a couple in colour, one of the gentleman performer, 
and one of the dance solo. Not good pictures as such, but quite 
suggestive of what being there looked like.[C][D]

The Kino ain’t fancy at all but it’s nice and friendly and the food and 
drink are just fine.

Oh, and Flamenco Sketches is the title of a really lovely tune, in a 
Spanish feel, on Miles Davis’ epochal Kind of Blue[5].

[1] http://www.xmlgrrl.com/blog/welcome/
[2] http://www.kinocafe.ca/
[3] /ongoing/When/200x/2008/04/13/K20D
[4] /ongoing/When/200x/2007/11/15/Sigma-30mm-F1-4
[5] http://en.wikipedia.org/wiki/Kind_of_Blue

[A] PS080894.png
[B] PS080895.png
[C] PS080910.png
[D] PS080922.png

Feed: ongoing
Filed under: Arts/Music/Performance, Arts, Music, Performance, Arts/Photos, Photos
****************************************
Item: WF2: Early Results
<WF-Early-Results#comments>
Date: 2008-06-09 05:16:29 UTC
[This is part of the Wide Finder 2[1] series.] The first serious results 
are in[2], and they’re eye-opening. The naive Ruby approach, remember, 
burned some 25 hours. There are now four other results posted, with 
elapsed times of 8, 9, 15, and 17 *minutes*. The write-ups are already 
full of surprises, and I’m expecting more.

[1] /ongoing/When/200x/2008/05/01/Wide-Finder-2
[2] http://wikis.sun.com/display/WideFinder/Results

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
****************************************
Item: Wide Finder 2
<Wide-Finder-2#comments>
Date: 2008-06-09 05:15:59 UTC
Last fall, I ran the Wide Finder Project[1]. The results were 
interesting, but incomplete; it was a real shoestring operation. I think 
this line of work is interesting, so I’m restarting it. I’ve got a new 
computer and a new dataset, and anyone who’s interested can play.

The Story Thus Far

I’ll use this entry as a table-of-contents for the series as it grows.

-

The Benchmark[2].
-

Benchmark Strawman[3].
-

First Pathetic Results[4].
-

People At Work[5].
-

Start Your Engines![6]
-

Early Results[7].

The Problem

The problem is that lots of simple basic data-processing operations, in 
my case a simple Ruby script, run like crap on modern many-core 
processors. Since the whole world is heading in the slower/many-core 
direction, this is an unsatisfactory situation.

If you look at the results from last time, it’s obvious that there are 
solutions, but the ones we’ve seen so far impose an awful complexity 
cost on the programmer. The holy grail would be something that maximizes 
ratio of performance increase per core over programmer effort. My view: 
Anything that requires more than twice as much source code to take 
advantage of many-core is highly suspect.

Last Time & This Time

There were a few problems last time. First, the disk wasn’t big enough 
and the sample data was too small (much smaller than the computer’s 
memory). Second, I could never get big Java programs to run properly on 
that system, something locked up and went weird. Finally, when I had 
trouble compiling other people’s code, I eventually ran out of patience 
and gave up. One consequence is that no C or C++ candidates ever ran 
successfully.

This time, we have sample data that’s larger than main memory and we 
have our own computer, and I’ll be willing to give anyone who’s 
seriously interested their own account to get on and fine-tune their own 
code.

The Set-Up

This time, the computer is a T2000[8], with 8 cores and 32 threads. It’s 
actually slower (well, fewer cores) than the semi-pre-production T5120 I 
was working with last time, but it’s got over 250G of pretty fast disks, 
which I’ve built a ZFS pool and one filesystem on.

I’ve also staged all the ongoing logfiles from February 2003 to April 
2008 there, 45G of data in 218 million lines.

It’s Internet-facing, with ssh access and port 80 open (not that I’m 
running a Web server yet).

Want to Join In?

I’ve set up a WideFinder Project Wiki[9]. The details of how to get 
started are there. For now, anyone with a wikis.sun.com account can 
write to it, which I’m hoping will be an adequate vandalism filter.

The First Step

Before we start coding, we need to agree on the Benchmark[10]; the 
13-line Ruby program was an instructive target for Wide Finder 1, but 
several people pointed out that this could be done with an awk/sort 
one-liner, so something a little more ambitious might be appropriate. 
I’ve made a couple of initial suggestions on the wiki.

[1] /ongoing/When/200x/2007/09/20/Wide-Finder
[2] /ongoing/When/200x/2008/05/12/WF-Benchmark
[3] /ongoing/When/200x/2008/05/22/WF-Benchmark-Strawman
[4] /ongoing/When/200x/2008/05/31/WF-First-Results
[5] /ongoing/When/200x/2008/06/02/WF-Others
[6] /ongoing/When/200x/2008/06/05/WF-Start-Me-Up
[7] /ongoing/When/200x/2008/06/08/WF-Early-Results
[8] http://www.sun.com/servers/coolthreads/t2000/index.xml
[9] http://wikis.sun.com/display/WideFinder/
[10] http://wikis.sun.com/display/WideFinder/The+Benchmark

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
****************************************
Item: WF2: Start Your Engines!
<WF-Start-Me-Up#comments>
Date: 2008-06-06 07:43:53 UTC
[This is part of the Wide Finder 2[1] series.] I have now done the first 
“official” run of the naive Ruby implementation of the benchmark. There 
is some discussion of the code here[2]. The benchmark is described, and 
the naive Ruby code provided, here[3]. I’ve started a results page 
here[4]. There are already ten eleven other people now with accounts on 
the Wide Finder machine, and I know there’ve been results that are 
hugely better than this first cut. Read on for a couple of notes on this 
first run.

The combination of using the SPARC-optimized Ruby and eliminating the 
huge sort shaved over four hours off the implementation time, but the 
results are still horrible: 25:24:41 elapsed time, with zero use of 
parallelism. The only thing that’s nice is the size: 78 lines of Ruby.

Can you do better on the time without exploding the code size?

An amusing note on the code. I needed a method to select the ten highest 
values (and corresponding keys) out of the hash table without sorting 
all of them. The code isn’t rocket science, here’s the skeleton:

class Hash
def top(howmany)
...
end
end
# ... later on
@clients = {}
...
@clients[client] += 1
...
top_clients = @clients.top(10)

I was showing it to a non-Rubyist and he said “WTF is that class Hash 
stuff?!” I explained and added, “Hmm, I guess I could have just said 
def top(hash, howmany)”. My questioner rolled his eyes. He had a point I 
think; Ruby makes you do some odd things.

[1] /ongoing/When/200x/2008/05/01/Wide-Finder-2
[2] /ongoing/When/200x/2008/05/31/WF-First-Results
[3] http://wikis.sun.com/display/WideFinder/The+Benchmark
[4] http://wikis.sun.com/display/WideFinder/Results

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
****************************************
Item: Everything On-Line?
<New-Yorker#comments>
Date: 2008-06-06 05:05:15 UTC
Recently I read The Rebellion Within[1] by Lawrence Wright, a long, 
erudite, immensely informative New Yorker piece about the internal 
dynamics of Al-Qaeda and the Muslim Brotherhood and Islamic Jihad: Who 
*are* these people, anyhow, where did they come from, and where are they 
going? I entirely recommend it. There’s a problem, though: I read it 
on-line.

A high proportion of my reading time these days, both business and 
pleasure, is spent reading the Internet. But we still subscribe to The 
Economist and, whenever I’m about to get on a plane, I try to pick up a 
copy of the New Yorker; its articles are mostly very good, and usually 
there are one or two which are long and meaty; just the thing for an 
airplane ride.

The problem was, on a recent trip, that I glanced at the cover of the 
latest, bought it there at the airport, and discovered only after I was 
airborne that the big meaty piece was that radical-Islamist story.

Lacking alternatives, I read most of it again anyhow once my laptop 
battery was juiced out. The experience, on paper, was immensely better. 
This is not to diss the New Yorker’s online presentation, as good as any 
there is; the column width is about perfect, and the cartoons are 
inlined artfully in a way that doesn’t impede the flow of reading.

But for now, paper’s just way better for this sort of long-form piece. 
Maybe that’ll change. I’m not sure what I’m actually going to *do* about 
this; I’ll hardly pass up a juicy link that I have time to follow just 
because it’s also on sale in print.

It’s just that there’s part of me that wishes that the online and print 
worlds were perhaps a bit more cleanly separated.

[1] http://www.newyorker.com/reporting/2008/06/02/080602fa_fact_wright

Feed: ongoing
Filed under: Business/Publishing, Business, Publishing, The World/Life Online, The World, Life Online
****************************************
Item: San Fran Shadows
<San-Fran-Shadows#comments>
Date: 2008-06-05 05:23:35 UTC
Last week I took a brief trip to San Francisco, and managed to escape 
for a walk. Lots of buildings had interesting shadows on them.[A][B][C]

The old parts of San Francisco seem to age with grace, by and large.

[A] PS080881.png
[B] PS080886.png
[C] PS080887.png

Feed: ongoing
Filed under: The World/Places/San Francisco, The World, Places, San Francisco, Arts/Photos, Arts, Photos
****************************************
Item: Not an OS
<Not-an-OS#comments>
Date: 2008-06-04 04:36:33 UTC
Last Thursday I tweeted[1]: “I strive to maintain an open mind when 
nontechnical people talk about the ‘Internet OS’ or ‘Web OS’. Sometimes 
it's tough.” I got some grumbles by email and I think the subject is 
worth more discussion. Let me be more specific: Neither the Internet nor 
the Web is much like an OS. And even if it were, that’d be the wrong way 
to think about what’s going on right now.

I’ve had this argument before. In March of 2007, Mike Arrington and I 
shared a stage at a startup event on the Sun campus, and talked about 
“Web 2.0”. Mike talked about a Web Operating System, I talked about 
people and information and business. Dan Farber reported ably[2]. I 
don’t think either of us convinced each other.

I: It’s About People Not Technology

The “Web OS” meme is harmful because it’s about technology. But the 
Internet’s killer app is people, has always been, will always be. Every 
single step forward has involved finding new routes and patterns and 
tools for people to use interacting with other people. No exceptions.

At the level of experience, Web 2.0 is about voices being heard. It used 
to be that there was a sanctified high priesthood that was licensed to 
publish: Journalists, novelists, and academics. Everyone else was an 
outsider, relegated to Letters-to-the-Editor or Speakers’ corner.

Nothing can possibly be more important than the experience of tens of 
millions of people, formerly outsiders, given a voice and an audience 
and a chance to join the global conversation.

This has been *technically* possible since there was an Internet. The 
crucial change was when Dave Winer and a few others proved that personal 
micro-publishing was fulfilling and self-supporting, by doing it.

Granted, a majority of people don’t publish—but if the world is even 
remotely like the population here at Sun, the knowledge that you’re 
empowered to do so is an attitude-changer.

Also granted, a majority of people who do publish don’t find big 
audiences. But that doesn’t matter; as Mena Trott was among the first to 
point out, “Publishing to fifteen people is perfectly OK if they’re the 
right fifteen people”.

II: It’s About Information Not Technology

Or to use another label, the Long Tail. We’re in a world where we expect 
a significant amount of the information and inspiration and creativity 
and criticism and collaboration and stories and pictures and music and 
movies to come from the many at the edge, not just the few at the 
center. Yes, there will always be a loud-voiced few at the center, but 
they’re no longer alone.

The traffic patterns have changed forever and they’re not changing back. 
Language and its flows are arguably one of the defining characteristics 
of /Homo sapiens/. The impact of all this information redirection on who 
we are and what we do is central. To focus on the technology is to miss 
the important point.

III: It’s About Business Not Technology

Lots of modern business is all about pumping information. The classic 
example would be finance; banks are giant information pumps with cash 
machines at the edges.

Organizations (business, governments, clubs, political parties, 
religions) who figure out how to surf the new information flow will 
succeed and prosper; those who push back will be swept away.

And it won’t have anything to do with whether anything’s like an OS or 
not.

IV: Nobody Uses the OS Anyhow

Suppose the Net or Web were OS-like. So what? Nobody programs to one of 
those any more. I bet if I asked a survey of 100 high-level professional 
developers what was the last time they actually coded a Unix or Linux or 
Win32 system call, I’d get a blank stare.

Everybody I know is, as a matter of principle, working in a higher-level 
programming environment that goes to great lengths to abstract away the 
OS: Java, Ruby, Apache-project C with APR, you name it.

V: It’s Platforms That Matter

When it comes to technology, what really matters is the platform you use 
to build with. Looking at your site from the outside, it’s not obvious 
to a non-expert (and perhaps not to an expert either) whether you built 
it with PHP or Rails or Django or JSF or ASP.NET.

But the choice of platform that you build with is incredibly important; 
in fact, most of the interesting new software innovations in the recent 
years have been in platform technology that let you build something good 
in less time with less pain that’s more maintainable.

The experience of building something with Rails is wildly different from 
that of a Java EE wrangler, which is wildly different still from that of 
someone in Microsoft-land. A common “Web OS”? Don’t see one.

In another sense, it’s reasonable to consider the Web as a platform; as 
Dave Winer memorably said “The platform that doesn’t have a vendor.” 
From the business and end-user and administrator point-of-view, this 
kind of works; but not from the seats where the people who build the 
apps sit.

In still another sense, it’s sane to refer to the browser as a platform; 
it’s certainly where a lot of code gets built. But it’s no more an OS 
than Gtk or WPF or Aqua is. It’s closest to the user but it’s a small 
piece of the big system.

The closest you’ll ever get to the essence of the Web-itself-as-platform 
is in working closely with HTTP and URIs, for example in implementing 
the Atompub protocol. But most developers working with Atompub will just 
use a handy library that fits into their Ruby or Java or C# ambiance, 
hardly even noticing.

VI: And Anyhow, It’s Not Like an OS

The OS is the software that sits between you and the hardware. In 
practice, it offers a set of brutally stupid and complex services for 
managing storage and networking and the lowest level of user 
interaction. It’s difficult and unpleasant to use.

I’ve used several and even contributed in a small way to one or two. 
Neither the Internet nor the Web are anything like one in any important 
way. Thank goodness.

[1] http://twitter.com/timbray/statuses/823088211
[2] http://blogs.zdnet.com/BTL/?p=4686

Feed: ongoing
Filed under: The World/Life Online, The World, Life Online, Business/Internet, Business, Internet, Technology/Internet, Technology, Technology/Web, Web
****************************************
Item: WF2: People At Work
<WF-Others#comments>
Date: 2008-06-03 15:16:51 UTC
[This is part of the Wide Finder 2[1] series.] I’m happy to report that 
I’ve given out a bunch of accounts on the Wide Finder 2 machine[2]. I’ll 
aggregate links to others’ work in this entry.

2008/06/02

Ray Waldin has a bunch of posts: WideFinder in Scala[3], WiderFinder[4], 
WideFinder 2 Strawman implemented in Scala[5], Multicore WideFinding[6], 
and Multicore Sorter[7].

Alex Morega has Wide Finder[8], Wide Finder: Going Parallel[9], and Wide 
Finder: Many Cores[10].

Did I Miss You?

If you’re working on this let me know and I’ll keep adding links.

[1] /ongoing/When/200x/2008/05/01/Wide-Finder-2
[2] http://wikis.sun.com/display/WideFinder/Infrastructure
[3] http://blog.waldin.net/2008/05/another-widefinder-in-scala_15.html
[4] http://blog.waldin.net/2008/05/widerfinder.html
[5] http://blog.waldin.net/2008/05/widefinder2-benchmark-in-scala.html
[6] http://blog.waldin.net/2008/06/multicore-widefinding.html
[7] http://blog.waldin.net/2008/06/multicore-sorter.html
[8] http://grep.ro/blog/2008/05/wide_finder
[9] http://grep.ro/blog/2008/05/wide_finder_going_parallel
[10] http://grep.ro/blog/2008/05/wide_finder_many_cores

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
****************************************
Item: Other Diddleys
<Other-Diddleys#comments>
Date: 2008-06-03 06:08:08 UTC
Gosh, Bo Diddley[1]’s gone. Which instantly reaffirmed one Internet 
lesson: if you want to boost the popularity of your YouTube videos, die. 
My two favorites (via Twitter today): Mona, with Tom Petty[2] (listen to 
Bo’s guitar) and then this ancient B&W TV shoot[3] with a young Bo and 
lots of screaming Sixties girls. But man, I love that rhythm, and lots 
of other people have played it well.

I’ve always smiled at George Thorogood’s[4] take on Bo’s Who Do You 
Love?[5], but the YouTube pickings are kind of slim; this[6] isn’t bad 
though it falls short of the version on 1978’s Move It On Over. But I 
personally think that Ronnie Hawkins[7] *owns* that song, and have 
always loved the Last Waltz performance[8]; for my money, one of Robbie 
Robertson’s best guitar outings ever.

And then there’s The Clash; around the time of London Calling[9] they 
may or may not have been the Best Rock Band in the World, but to me at 
least, they were The Only Band That Mattered.

And on London Calling there were *two* bo-diddleys right after each 
other: Hateful and Rudie Can’t Fail. There’s a lot of Clash on YouTube, 
and watching some now I’m reminded of how unbelievably great they were 
live. But I can’t find either of those tunes. (Well, there’s a very 
decent Rudie[10] from Joe Strummer and the Mescaleros). On the 
totally-unlikely off-chance that you like rock but don’t have London 
Calling, drop everything and buy it now.

Thanks for that rhythm, Bo, and those tunes built around it. There’ll be 
someone playing a bo-diddley in a bar somewhere as long as /Homo 
sapiens/ draws breath.

[1] http://en.wikipedia.org/wiki/Bo_Diddley
[2] http://youtube.com/watch?v=YVRtQUTd7Gk
[3] http://www.youtube.com/watch?v=zBAJXyF1HVc
[4] http://en.wikipedia.org/wiki/George_Thorogood
[5] http://en.wikipedia.org/wiki/Who_Do_You_Love
[6] http://youtube.com/watch?v=PYmMhnM-y4Q
[7] http://en.wikipedia.org/wiki/Ronnie_Hawkins
[8] http://youtube.com/watch?v=dn0SngXfzKk
[9] http://en.wikipedia.org/wiki/London_Calling
[10] http://www.youtube.com/watch?v=QsZUBpNLGAU

Feed: ongoing
Filed under: Arts/Music, Arts, Music
****************************************
Item: Enterprisey Goodness
<JBI#comments>
Date: 2008-06-02 19:15:57 UTC
I tend to liking simple things, and to suspicion of big “enterprisey” 
software frameworks. I am dimly aware that up in the clouds there are 
platforms built on platforms built on platforms built on Java, towering 
edifices where acronyms like “JBI” and “ESB” and “SCA” live. Except for, 
I could never figure out what they actually, you know, *did*. Let’s be 
honest; the complexity and the high-level arm-waving about “Integration” 
scared me away and I never really tried. Well, I’ve stumbled into a 
closer look and am beginning to think there’s some there there.

I’ve been talking to Andi Egloff[1], who lives up there among the 
floating acronyms in these enterprisey clouds. I asked “Well, what does 
this software actually *do*?” After some talk he said “Let me show you 
what we’re working on.” Showing is so much better than telling.

They’re calling their work-in-progress “Fuji”. Here’s a screencast[2] 
where they hook together an RSS feed, an XMPP stream, and a log-file, 
with an interposing a Ruby filter, using Fuji and a bit of DSL voodoo.

They’re also thinking of a visual interface; here’s the same logic[3], 
only with little boxes and arrows instead of DSLs. Personally I prefer 
the DSL.

It’s all based on OSGI[4], hmmm.

Now, in a world that already has Yahoo Pipes[5] and simple Ruby 
POP-to-Twitter mashups[6], one wonders if you need a big honking Java 
howitzer to squish this particular ant.

So I asked Andi that, and he said “Yes, but then with JBI you can also 
talk to ESBs and other traditional enterprise integration points”. Which 
I’m not sure that YPipes or little Ruby scripts are built for. I asked 
him to show me that and he said they’re going to. I’m interested; stay 
tuned.

[1] http://blogs.sun.com/andi/
[2] http://blogs.sun.com/andi/entry/bringing_together_jbi_osgi_and
[3] http://blogs.sun.com/andi/entry/fuji_gui_sneak_peek_fingers
[4] http://blogs.sun.com/andi/entry/why_is_osgi_on_everyone
[5] http://pipes.yahoo.com/pipes/
[6] /ongoing/When/200x/2008/04/30/PDML-Twitter-Mashup

Feed: ongoing
Filed under: Technology/Java, Technology, Java, Technology/Web, Web
****************************************
Item: WF2: First Pathetic Results
<WF-First-Results#comments>
Date: 2008-05-31 21:24:46 UTC
[This is part of the Wide Finder 2[1] series.] I made a couple of little 
changes to the Strawman Benchmark[2] and let ’er rip on the big 45Gig 
data set. The results were miserable, but instructive.

Just How Miserable?

29:31:30 elapsed, 29:30:52 CPU, i.e. it maxed one of the T2000’s 32 
CPUs. That’s one *day* plus five hours and change. Ouch!

Small lessons

First of all, it was just using Ruby-by-default-out-of-the-box. As Haik 
pointed out[3], there’s a SPARC-optimized version available from Sun and 
I’ll install that.

Second, the code computes the various top-10 lists (hits, bytes, 
clients) by storing the numbers in a hash and running 
Enumerable#sort_by. It says right in the docs for that function that 
it’s inefficient. Still, it wasn’t too bad for sorting hashes keyed by 
ongoing resources, because there are only a few thousand of those. But 
when it sorted the hash keyed by the hostnames that had fetched 
resources, well... it wasn’t pretty. The sort_by call took hours.

It was made worse by a programming error on my part; several people 
pointed out that one couldn’t diff the program output because if several 
of the top entries had the same numeric value, they could show up in 
arbitrary order. I simply changed the code to sort on the value, then 
the key. Of course, I stupidly didn’t take out the old sort, so it ended 
up sorting twice.

Anyhow, I’ve changed the code to do a single O(N) scan through the big 
hashes, rather than sort them.

Big Lessons

The naive Ruby code is making terrible use of the machine. Over the 
course of the run, it uses, on average, 3.2% of the available CPU 
resources, and 3.8% of the available memory.

If you look at the Bonnie results[4], this sucker can do simple block 
I/O at about 150M/second at a cost of about 75% of one CPU. So you ought 
to be able to plow through the 45G dataset in under five minutes, 
leaving 31 CPUs to do the actual work, even without fancy mmap() tricks. 
I bet somebody will write a WF2 implementation that runs in about five 
minutes as opposed to 29 hours.

Next Steps

I’m going to re-run the cleaned-up Ruby code with the SPARC-optimized 
MRI and propose that as the baseline benchmark for WF2 implementations 
to beat. Then the fun should start.

[1] /ongoing/When/200x/2008/05/01/Wide-Finder-2
[2] /ongoing/When/200x/2008/05/22/WF-Benchmark-Strawman
[3] 
http://www.tbray.org/ongoing/When/200x/2008/05/22/WF-Benchmark-Strawman#c1211521663.324677
[4] http://wikis.sun.com/display/WideFinder/Infrastructure

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
****************************************
Item: Family Moment
<Discovery#comments>
Date: 2008-05-31 06:43:51 UTC
The back porch door was open and our adorable little 
almost-two-year-old, all golden curls and pink dimples, was wandering in 
and out. One time as she was coming in, I noticed Marlowe the cat was 
sort of hovering around her; then the air was full of female shrieks: 
“She’s got a *dead rat*!” And so she did, holding it up all eager to 
please, while Marlowe looked confused and irritated. A fair-sized one 
too, with a good five inches of dangling tail. A few seconds later, she 
was just as confused and irritated as Marlowe, for the same reason, and 
the corpse was headed for disposal.

(Well, yes, we have two healthy cats that go outside, and this is a 
waterfront city thus will always have rats, and our cats are well-fed so 
they bring their prizes home to the back porch to show off.)

Feed: ongoing
Filed under: The World/Family, The World, Family, The World/Humor, Humor
****************************************
Item: WF2: The Benchmark
<WF-Benchmark#comments>
Date: 2008-05-31 06:19:43 UTC
[This is part of the Wide Finder 2[1] series.] A bunch of people have 
requested the sample data. Meanwhile, over at the wiki[2], there’s a 
decent discussion going on of what benchmark we should run.

I particularly liked the comments from MenTaLguY[3], Preston 
Bannister[4], and Erik Engbrecht[5]. (Unfortunately, given the test data 
we have, I don’t see us being able to follow Erik’s “non-ASCII” 
suggestion).

Of the suggestions on offer at the benchmark page[2], I prefer the last 
couple. Session statistics are a little trickier, and I suspect a bit 
more resistant to a brute-force Map/Reduce approach. And the “normal 
HTML report” idea is very much in the spirit of Wide Finder 1, only with 
a bit more computation involved, perhaps enough to keep it from being a 
pure parallel-I/O benchmark.

Your thoughts? I suppose I’m signed up to build a reference 
implementation for verifying output.

[1] /ongoing/When/200x/2008/05/01/Wide-Finder-2
[2] http://wikis.sun.com/display/WideFinder/The+Benchmark
[3] 
http://wikis.sun.com/display/WideFinder/The+Benchmark?focusedCommentId=25887354#comment-25887354
[4] 
http://wikis.sun.com/display/WideFinder/The+Benchmark?focusedCommentId=25887368#comment-25887368
[5] 
http://wikis.sun.com/display/WideFinder/The+Benchmark?focusedCommentId=25887443#comment-25887443

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
****************************************
Item: WF2: Benchmark Strawman
<WF-Benchmark-Strawman#comments>
Date: 2008-05-31 06:19:35 UTC
[This is part of the Wide Finder 2[1] series.] I’ve just updated the 
Wide Finder wiki[2] with a straw-man benchmark (see “Strawman”), with 
sample Ruby code and output. Argh... it takes Ruby over 23 seconds to 
process just 100K lines. I hear talk of a Solaris/SPARC optimized 
version, gotta track that down. Comments please on the benchmark, then 
we can all start bashing our heads against the big file.

[1] /ongoing/When/200x/2008/05/01/Wide-Finder-2
[2] http://wikis.sun.com/display/WideFinder/The+Benchmark

Feed: ongoing
Filed under: Technology/Concurrency, Technology, Concurrency
