Open eyes Working brain

So, Google is going to kill Wave.

But despite these wins, and numerous loyal fans, Wave has not seen the user adoption we would have liked. We don’t plan to continue developing Wave as a standalone product, but we will maintain the site at least through the end of the year and extend the technology for use in other Google projects.

That’s a great example of why “do one thing well” is better than “do it all”. Not only because, as Gall’s law implies, it’s easier to get a simple system straight, but also because once you get a complex system straight, you still have to make people get their mind around it. Wave made chat and e-mail play well together, with a great replay feature, bots and translation. Twitter gives short text updates. Adoption declared the winner, but we already knew it.

I spoke at Better Software 2010, together with Fullo, about speeding up web applications. The talk draws heavily from Steve’s work, but it’s a little bit different from current literature because it tries to organize best practices not as flat list but under macro-areas emerged as “slow parts”. Also, i concluded with my obsession that complexity inherently introduced by performance optimizations should not be dealt with by programmers directly, but by means of automation and abstraction.

Here it is.

update: now i am linking to the extended version which i gave at phpday 2010

We live in times of complexity, and even though neat technologies and elegant software can be found at times, the market is still definitely dominated by absurdly heavy solutions. Enterprise is imploding and a wind of change towards more sustainable approaches is blowing all around us, yet the mainstream scene is comparatively stagnant and all the pain inflicted to people is not really causing the deserved rebellion.

Why is that? Why when confronted by the possibility of rewriting their untestable bloatware, customer’s reply is almost always invariably “No, we don’t need it. We’ll just have to fix known bugs and add a couple of features, because right as it is, the software works ok…”?. What does “works ok” really mean? In my experience, it translates roughly to “The software does not physically blows up our office, it does some of the things we need to do, and over the years our employees have developed a thick skin against all the nuisances and a baggage of manual tricks, passed on by mouth, to get the rest of the work done anyway. Oh, and we already paid a lot for it”.

Recently, i got a taste of this mindset myself, when i booked online 2 tickets to Avatar at the local cineplex

“Hello this is my reservation code”

“Sorry Mr, those seats are reserved”

“Sure, by me”

“No, actually by others”

“What? see, i made this online reservation…”

“I see, but we take reservations both online and by phone, sometimes they overlap and phone is given priority”

“Overlap?! No trust me, i am a programmer, overlapping reservations are not supposed to happen, because your system has to take care”

“Oh, but evidently it doesn’t”

“WTF?!?!”

“Please, don’t get mad, i am gonna give you other seats. Today is not even bad. You should see how many angry people we must manage during christmas holidays when all movies are sold out!”.

Now, given that reservation means “An arrangement by which accommodations are secured in advance”, how would you rate a reservation system that does not guarantee secure accommodations? Like a fish unable to breathe underwater, yet they live with it, and this takes me to the point.

First, humans are best when it comes to adaptation. That means we naturally adapt to pain so that we don’t feel so bad, and adapt to pleasure so that we don’t feel so good. Perception of any external stimulus in the end comes to balance. Barry Schwartz in the Paradox of Choice says:

respondents were asked to rate their happiness on a 5-point scale. Some of them had won between $50,000 and $1 million in state lotteries within the last year. Others had become paraplegic or quadriplegic as a result of accidents. Not surprisingly, the lottery winners were happier than those who had become paralyzed. What is surprising, though, is that the lottery winners were no happier than people in general. And what is even more surprising is that the accident victims, while somewhat less happy than people in general, still judged themselves to be happy.

Second, humans are also very bad at admitting sunk costs. The idea of having spent money on something not worth is the ultimate inconvenient truth. Again Barry

Aversion to losses also leads people to be sensitive to what are called “sunk costs.” Imagine having a $50 ticket to a basketball game being played an hour’s drive away. Just before the game there’s a big snowstorm—do you still want to go? Economists would tell us that the way to assess a situation like this is to think about the future, not the past. The $50 is already spent; it’s “sunk” and can’t be recovered. What matters is whether you’ll feel better safe and warm at home, watching the game on TV, or slogging through the snow on treacherous roads to see the game in person. That’s all that should matter. But it isn’t all that matters. To stay home is to incur a loss of $50, and people hate losses, so they drag themselves out to the game.

Third, as brilliantly pointed out by Ryan Brush’s “Code is Design” in 97 Things Every Programmer Should Know and by Gabriele’s “Waterfall Pitfall #1″ (italian), uninformed most people understand software construction in terms of the better known building construction. Now, since programs are built out of bytes (not bricks), which are practically nothing, using mind (not excavators), which has no physical constraints, actual construction must be very cheap. This gives them the false hope of having an easy exit strategy at their disposal: fixing the software when an emergency comes up. Would they wait for a defective bridge to show the first cracks before attempting to fix it? Their unconstrained minds seem to be unable to realize that story construction aka book writing, built out of words, might represent a more fitting comparison and that The Divine Comedy took Dante, a renowned genius, more than ten years to finish.

Last but not least, mainstream has made a really good job at covering mistakes of incompetent programmers. From the almost sandboxed life cycle of a php script, to the rigid syntax of java and its self-correcting IDEs, to the plethora of useless certifications, great efforts have been devoted to make any primate with opposable thumbs able to program with very limited competence. Many and cheap, that’s how economy of scale is supposed to fail work, and that’s how we got this horde of unprofessional programmers sacking the best projects.

All of these points help to explain proliferation of crappy software. Maybe, they get it from some body rental which pays more for advertising than for the army of juniors that actually does the job. At the beginning it hurts, but they spent good money and cannot afford to accept failure, so lies are told and more time and money are invested to improve the situation. Then workarounds, albeit inefficient, come and direct suffering somehow decreases. Eventually, the pile of workarounds becomes part of company culture, and all is back to balance: the software starts working ok.

Unfortunately, this means that the quest for better software workflows can hardly come out of necessity, it must come out of vision, and vision takes inspiration fed to working brains then time for the masses to catch up. With Universe hopefully taking care of latter two, i like to think we, professional programmers, are those in charge of the former.

The browser is emerging as the best platform for applications, so a large community is growing to address its final weakness: speed. Google, Yahoo and various independent programmers are all pushing a bunch of clever techniques to boost performance and please end users. That’s nice, yet as Mark Twain once said, half of the results of good intentions are evil and i see potential danger in many of the suggestions made. Here a representative short list of them:

• Avoid for-in and forEach in favor of optimized while loops
• Before making modifications to a DOM node remove it and then re-insert it
• To insert multiple DOM nodes, first insert them into a Document Fragment and then add it to the DOM
• Join all scripts into a single file
• Load javascript files on demand

Let’s make it clear for once, execution speed is not a human problem, that’s what computers are for, they execute our commands fast. The human problem is programming speed and writing down clear, readable, maintainable commands aka programs. forEach loops make sense to me, they say “i want to do something on each item”, optimized while loops make sense to computers. If i want to add DOM nodes or modify one, i don’t care of removing it or document fragments, browsers care. To me it’s just noise. Almost all of the problems addressed by those techniques stem from lack of smartness in the browser, and that’s where fixes belong to, on the machine side. The fact that there are inept browser makers is no excuse. Fixes still belong to the machine, they’re repeatable and can be made automatic. We have a long history of programs automatically converting human friendly code to machine friendly code. They’re called compilers and the output either machine code or optimized javascript doesn’t matter.

So, learn about javascript performance since knowledge is always the way, but don’t turn yourself into a machine, you’d be an awful one. Use the tools and wait for browsers to catch up.

• Aptimize
• Closure Compiler
• YUI 3 use

Google Wave is an impressive product. If you haven’t seen the Google IO Preview video, i highly suggest you do it know and come back later. Even if you don’t care, there’s a long list of smart ideas, patterns and neat technical solutions packed into Wave that deserve to be studied. Let me mention a few:

• One unique state, on the server
• Operational Transformations
• Statistical spell checker
• Real time collaboration
• Real time search
• Federation
• Playback of changes over time
• Applications as Robots
• Openness of protocol, api, extension

and much more, but even though all this goodness and google involvement are exciting and indeed already suggest a planetary success, i am a little skeptical.

First, in our times of overwhelming complexity, when products like Twitter can WIN not by augmenting capabilities but by restricting them (to a single short answer), Wave  rich complexity will have to prove itself. But there’s more. Something feels wrong to me in this catch-all approach to communication, and i came to believe it’s how all this empowerment is unevenly distributed among different facets of communication itself.

Let me explain trying to determine a few of those facets:

• Retention. How long information will remain available. e.g. From ephemeral spoken words to durable written words.
• Responsiveness. Whether information is taken in immediately or at a later time. e.g. From asynchronous mail to synchronous phone.
• Relevance. How much information is specific and important to a certain matter or goal. e.g. From generic chit-chat to focused articles.

It’s easy to see that high retention and asynchronicity are symbiotic. There can be no asynchronous transfer of information if that information is not stored somewhere waiting to be consumed. What’s harder to see, but empirically true afaic, is that high relevance is related as well. Maybe it’s the effort traditionally required to store information in a durable form, maybe it’s because people have more time to think on what they want to say, maybe it’s that information is produced in isolation helping to concentrate, but it seems asynchronous highly persistent communication tends to be more focused, worth and relevant.

• You’d write a book about origami, but probably not about your last vacation.
• You’d write a blog post about your last vacation, but maybe not about today’s weather.
• You’d twitter about weather, but hopefully not that you’re turning down volume on the ipod.
• Yet you could say that to a friend being around.

Conversely, as communication responsiveness bar raises up, we keep putting in more and more trivial, irrelevant, conversational stuff. That’s easily common sense. Take the word “chat” and how it often has negative connotation. Chat as a waste of time. Chat as enemy of focused productive work (pomodoro technique anyone?). It would be foolish to say that all real time communication is irrelevant, or that irrelevant information is always bad, yet down that path, we often digress from the meaningful and, in that circumstance, high retention is at least useless if not undesirable.

So what’s the problem with Google Wave?

It innovates by opening the entire spectrum of responsiveness possibilities. Users can communicate from asynchronous e-mail to true real time chat. Wonderful. But what about other facets? 

• Retention is fixed and very high. In fact so high that waves can be viewed as conversations or even documents. Stored forever in the cloud. I guess they have or will have a delete button but that would be quite coarse-grained anyway.
• Relevance has even weaker support. You have wave title and tags and that’s it.

Without further control, i don’t see how people will be kept from mixing those different kinds of communication in wrong ways, especially as number of participants goes up. I imagine carefully built documents polluted by real time blabbering, collaborative flame wars producing endless amounts of noise and whole branches of waves slipping out of topic. Trivial fast-paced conversations and relevant slowly crafted documents, all together in a persistent rich beautiful mess called the wave.

I hope to see odds of such hellish prophecy mitigated by some good moderation system and/or by giving better control on retention, otherwise Wave may become yet another perfect illustration of “giving enough rope”. In conclusion, some pretty tentative attempts to contribute:

• an additional chat system orthogonal to wave content
• tags and votes on content and ability to show/hide based on that
• content that fades away unless marked “important”
• content time to live