Archive for the ‘programming’ Category

Die Semicolon Die!

Wednesday, May 25th, 2011

I’ve always put javascript’s automatic semicolon insertion (ASI for short) under the bad parts of the language. That is based on Douglas Crockford’s explanation of how the feature is tricky and easily leads to mistakes, with the canonical example being:

// good, returns the object
return { ... }

// wrong! returns undefined
return
    {
        ...
    }

Fair enough. Lately i’ve been doing more and more ruby. Ruby is a language universally praised for its elegant, easy to read syntax. One of the strong points of the syntax is its terseness, that is, you can omit a lot of punctuation. Semicolons as well. Wait a moment…

def test
    return
        {
            ...
        }
end

test # returns nil !!

Same thing! Having the meaning of a program change due to an end-of-line is not a good thing in ruby as well, but it’s widely accepted because of the benefits. This must be true for javascript as well, so first point:

“Removing semicolons and other punctuation clutter is not just a liability. It actually makes your code look better.”

So both the languages have to decide when a statement implicitly terminates. But is ruby implementation really the same as javascript? It turns out it’s not, ruby takes a quite safer approach. A statement in ruby is finished on an end-of-line if it’s syntactically valid by itself, it spans multiple lines if it’s not:

# this works, the trailing dot means the statement is not finished
object.
    method1.
    method2.
    method3

# syntax error, first line is a valid statement by itself, second line calls method1 on nothing
object
    .method1
    .method2
    .method3

It’s safe because how a line is parsed depends on the line itself, not by other lines that could be written “by others”. The bad part is how it makes method chaining on multiple lines look ugly. This is why ruby 1.9 introduced the exception “the statement continues if the first character of next line is a dot”.

Javascript takes a step further to solve this bad part. A controversial step. A statement is finished on an end-of-line if the first character of the next line cannot be correctly parsed as if it was part of the line. Otherwise, the statement goes on. This removes the clutter and gives nice chaining:

// just works
object
    .method1()
    .method2()
    .method3()

Unfortunately, you now have a nasty problem. 2 lines which are supposed to be 2 different statements, but with the first character of the second line being a valid continuation of the first, will be treated as one statement with unpredictable results. This practically happens only when a line starts with either ( [ + - /

// function call instead of grouping
var a = b + c
(d + e).print()
// is really
var a = b + c(d + e).print()

// array index instead of array literal
var a = ["a", "b", "c"]
[0, 1].forEach( … )
// is really
var a = ["a", "b", "c"][0, 1].forEach( … )

// binary math operator instead of unary
var a = b + c
-1 == string.indexOf(query) || die()
// is really
var a = b + c – 1 == string.indexOf(query) || die()

// division instead of regular expression
var i=0
/[a-z]/g.exec(s)
// is really
var i=0 /[a-z]/g.exec(s)

Well, this sucks, so what should you do? I could say that i remember being caught by this problem just once in many years of javascript. The return problem or starting a line the nasty way is something extremely rare. But even if you don't want to afford the risk, why avoid ASI without even knowing about it? Without even thinking about a reasonable fix, given the nicer syntax? And this leads me to the second point:

"To write semicolon-free code and avoid getting bitten, you just need to remember 2 rules

1) Don't put an end-of-line between return, break, continue, throw, postfix ++, postfix -- and their operand
2) Avoid starting a line with ( [ +  - / but if you have to, prepend it with a semicolon"

// everything's fine
return { ... }
continue label
break label
throw error
counter++
counter--

var a = b + c
;(d + e).print()

var a = ["a", "b", "c"]
;[0, 1].forEach( ... )

var a = b + c
;-1 == string.indexOf(query) || die()

var i=0
;/[a-z]/g.exec(s)

Is it that taxing to remember? Automatic semicolon insertion is of course controversial, but using it is not a complete failure. It's a matter of taste, a trade-off between cleaner nicer code and some tough albeit avoidable pitfall.

While i'm at it, let's debunk some well known myths that always show up

  • "I could know ASI but others don't and they will mess things out"
    Well this may be true. It depends on where you work, the skill of your peers, etc.. To me, a javascript programmer is just supposed to know this stuff as he knows of prototype and first class functions. If they don't, supposing they got the opposable thumbs, as they can be told to put semicolons everywhere, they can be told to remember the above 2 simple rules.
  • "It's not gonna work the same way on every browser"
    It's in the specs since more than a decade. I think browser bugs are a thing of the past and even proponents of this theory look unable to find something newer than 5 years ago, so.
  • "It breaks the tools. You cannot minify code anymore, etc..."
    Let's be clear about this. It's officially part of the language. A tool unable to cope with ASI is a broken tool, period. Anyway, i have never had a problem with google closure compiler.
  • "Jslint doesn't work with it"
    Jslint enforces the vision of Douglas and it's pretty strict about it. This is fair, yet for those having another vision nothing is wrong with using Jshint which has an option to accept ASI. 

Let's close with two very nice articles that explain the details and of course you can always read the ecmascript specs:

The most well-written comprehensive article

Very good explanation of the pitfalls

The plain specs

A Lot of Javascript Love

Friday, November 12th, 2010

I am back from Webtech Conference Italia 2010. One of the first in Italy featuring a full javascript day with six talks. Not counting javascript related talks in other tracks. It has been exciting to see javascript explained in patterns, historically and computationally analyzed, tuned for faster websites, organized in popular libraries, used to query modern databases, to extract data from the web, to mashup those data, to program mobile devices, improved in latest browsers and at last on the server side to build scalable, fast network applications.

We’re witnessing the exponential rise of a neglected little broken language. A language that obviously started with something very right and grew up even better.

A language that is finally gonna get A LOT of love.

P.S. Here are my talks:

Please Don’t Touch the Slow Parts

Saturday, May 8th, 2010

I spoke at Better Software 2010, together with Fullo, about speeding up web applications. The talk draws heavily from Steve’s work, but it’s a little bit different from current literature because it tries to organize best practices not as flat list but under macro-areas emerged as “slow parts”. Also, i concluded with my obsession that complexity inherently introduced by performance optimizations should not be dealt with by programmers directly, but by means of automation and abstraction.

Here it is.

update: now i am linking to the extended version which i gave at phpday 2010

All Software Works Ok

Wednesday, March 31st, 2010

We live in times of complexity, and even though neat technologies and elegant software can be found at times, the market is still definitely dominated by absurdly heavy solutions. Enterprise is imploding and a wind of change towards more sustainable approaches is blowing all around us, yet the mainstream scene is comparatively stagnant and all the pain inflicted to people is not really causing the deserved rebellion.

Why is that? Why when confronted by the possibility of rewriting their untestable bloatware, customer’s reply is almost always invariably “No, we don’t need it. We’ll just have to fix known bugs and add a couple of features, because right as it is, the software works ok…”?. What does “works ok” really mean? In my experience, it translates roughly to “The software does not physically blows up our office, it does some of the things we need to do, and over the years our employees have developed a thick skin against all the nuisances and a baggage of manual tricks, passed on by mouth, to get the rest of the work done anyway. Oh, and we already paid a lot for it”.

Recently, i got a taste of this mindset myself, when i booked online 2 tickets to Avatar at the local cineplex

“Hello this is my reservation code”

“Sorry Mr, those seats are reserved”

“Sure, by me”

“No, actually by others”

“What? see, i made this online reservation…”

“I see, but we take reservations both online and by phone, sometimes they overlap and phone is given priority”

“Overlap?! No trust me, i am a programmer, overlapping reservations are not supposed to happen, because your system has to take care”

“Oh, but evidently it doesn’t”

“WTF?!?!”

“Please, don’t get mad, i am gonna give you other seats. Today is not even bad. You should see how many angry people we must manage during christmas holidays when all movies are sold out!”.

Now, given that reservation means “An arrangement by which accommodations are secured in advance”, how would you rate a reservation system that does not guarantee secure accommodations? Like a fish unable to breathe underwater, yet they live with it, and this takes me to the point.

First, humans are best when it comes to adaptation. That means we naturally adapt to pain so that we don’t feel so bad, and adapt to pleasure so that we don’t feel so good. Perception of any external stimulus in the end comes to balance. Barry Schwartz in the Paradox of Choice says:

respondents were asked to rate their happiness on a 5-point scale. Some of them had won between $50,000 and $1 million in state lotteries within the last year. Others had become paraplegic or quadriplegic as a result of accidents. Not surprisingly, the lottery winners were happier than those who had become paralyzed. What is surprising, though, is that the lottery winners were no happier than people in general. And what is even more surprising is that the accident victims, while somewhat less happy than people in general, still judged themselves to be happy.

Second, humans are also very bad at admitting sunk costs. The idea of having spent money on something not worth is the ultimate inconvenient truth. Again Barry

Aversion to losses also leads people to be sensitive to what are called “sunk costs.” Imagine having a $50 ticket to a basketball game being played an hour’s drive away. Just before the game there’s a big snowstorm—do you still want to go? Economists would tell us that the way to assess a situation like this is to think about the future, not the past. The $50 is already spent; it’s “sunk” and can’t be recovered. What matters is whether you’ll feel better safe and warm at home, watching the game on TV, or slogging through the snow on treacherous roads to see the game in person. That’s all that should matter. But it isn’t all that matters. To stay home is to incur a loss of $50, and people hate losses, so they drag themselves out to the game.

Third, as brilliantly pointed out by Ryan Brush’s “Code is Design” in 97 Things Every Programmer Should Know and by Gabriele’s “Waterfall Pitfall #1″ (italian), uninformed most people understand software construction in terms of the better known building construction. Now, since programs are built out of bytes (not bricks), which are practically nothing, using mind (not excavators), which has no physical constraints, actual construction must be very cheap. This gives them the false hope of having an easy exit strategy at their disposal: fixing the software when an emergency comes up. Would they wait for a defective bridge to show the first cracks before attempting to fix it? Their unconstrained minds seem to be unable to realize that story construction aka book writing, built out of words, might represent a more fitting comparison and that The Divine Comedy took Dante, a renowned genius, more than ten years to finish.

Last but not least, mainstream has made a really good job at covering mistakes of incompetent programmers. From the almost sandboxed life cycle of a php script, to the rigid syntax of java and its self-correcting IDEs, to the plethora of useless certifications, great efforts have been devoted to make any primate with opposable thumbs able to program with very limited competence. Many and cheap, that’s how economy of scale is supposed to fail work, and that’s how we got this horde of unprofessional programmers sacking the best projects.

All of these points help to explain proliferation of crappy software. Maybe, they get it from some body rental which pays more for advertising than for the army of juniors that actually does the job. At the beginning it hurts, but they spent good money and cannot afford to accept failure, so lies are told and more time and money are invested to improve the situation. Then workarounds, albeit inefficient, come and direct suffering somehow decreases. Eventually, the pile of workarounds becomes part of company culture, and all is back to balance: the software starts working ok.

Unfortunately, this means that the quest for better software workflows can hardly come out of necessity, it must come out of vision, and vision takes inspiration fed to working brains then time for the masses to catch up. With Universe hopefully taking care of latter two, i like to think we, professional programmers, are those in charge of the former.

How I did It: Touch Typist in five months

Wednesday, January 27th, 2010

it-could-workUntil, from the midst of this darkness, a sudden light broke in upon me, a light so brilliant and wonderous, and yet so simple. Deep practice, eradicate errors and deep practice again. I alone succeeded in discovering the secret of bestowing skill. Nay, even more, I myself became capable of bestowing mastery upon apprentice matter.

It could work.

At the end of august 2009, while i was reading about the importance for a programmer to touch type, i found out many programmers could type at 80 wpm and above. I tried myself and consistently scored 60 wpm or below. I have been spending many hours a day at the keyboard since at least 1996. How could it be i am not as fast? I already knew about the importance of practice, but i did practice for years, didn’t I? Well, uhm, no.

Think of the career of a professional football (soccer) player. How does he get there, does he play all the time for years? Actually, he spends most of the week training: stretching, pushups, sprints, weights, long runs, ball work, etc… Only a small percentage of time is indeed to play short games and the main game on sunday. That’s because play alone can only push your performance to the upper end of the range set by current skill. It’s not going to push you to the next level, to the next order of magnitude, and it doesn’t keep you from developing bad habits. And that’s where i got, very fast at typing in my very flawed 7-fingers posture. Not able to improve any further, not even in more than 10 years.

I really wanted to get better, so the questions were How? and How long will it take? I didn’t know the answer to the latter but the former was by then clear in my mind. Find a keyboard dojo, do keyboard katas and take the needed time. After trying a lot of viable solutions i found my dojo at www.typingweb.com. To add some salt to the challenge i switched keyboard layout from italian to U.S., which is quite better for programming, and i started practicing daily with their courses, routinely taking tests to record my progress. Two pomodoros a day for the first two months, then one, then again two when i was reaching the end. Always striving to get 97% or above accuracy at the higher possible speed for every single lesson.

Now, five months and about 120 pomodoros of deep practice later, i am writing this to shout to the world that IT COULD WORK. Next i’ll be moving to vim katas, while still having some fun at www.typeracer.com, because you never really stop to practice right?!

Yours sincerely,
ten fingers typist
american layout
80 wpm average
up to 100 wpm under a good moon
Federico.

typing_sep09

typing_nov09-gen10

Javascript Performance: Make the Browser Happy (and You Sad)

Sunday, November 15th, 2009

BENDERThe browser is emerging as the best platform for applications, so a large community is growing to address its final weakness: speed. Google, Yahoo and various independent programmers are all pushing a bunch of clever techniques to boost performance and please end users. That’s nice, yet as Mark Twain once said, half of the results of good intentions are evil and i see potential danger in many of the suggestions made. Here a representative short list of them:

  • Avoid for-in and forEach in favor of optimized while loops
  • Before making modifications to a DOM node remove it and then re-insert it
  • To insert multiple DOM nodes, first insert them into a Document Fragment and then add it to the DOM
  • Join all scripts into a single file
  • Load javascript files on demand

Let’s make it clear for once, execution speed is not a human problem, that’s what computers are for, they execute our commands fast. The human problem is programming speed and writing down clear, readable, maintainable commands aka programs. forEach loops make sense to me, they say “i want to do something on each item”, optimized while loops make sense to computers. If i want to add DOM nodes or modify one, i don’t care of removing it or document fragments, browsers care. To me it’s just noise. Almost all of the problems addressed by those techniques stem from lack of smartness in the browser, and that’s where fixes belong to, on the machine side. The fact that there are inept browser makers is no excuse. Fixes still belong to the machine, they’re repeatable and can be made automatic. We have a long history of programs automatically converting human friendly code to machine friendly code. They’re called compilers and the output either machine code or optimized javascript doesn’t matter.

So, learn about javascript performance since knowledge is always the way, but don’t turn yourself into a machine, you’d be an awful one. Use the tools and wait for browsers to catch up.

Google Test Automation Conference: Testing is not enough

Monday, October 26th, 2009

niklaus wirth at google

If i had to award the best talk at GTAC 2009, the no-brainer choice would be Prof. Niklaus Wirth opening talk. That’s not surprising if you consider who the speaker is, one of the great pioneers of computer science in the field of programming languages. What’s most surprising to me is that he presented a (pre)historical review of problems which turned out to be incredibly relevant today and, somehow, forced me to reframe my understanding of testing.

Building on 1972 Dijkstra dismissal of software testing

program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence.

Wirth explained that testing is treating symptoms instead of the disease, with the disease being our failure to prove correctness of programs by analytical means. This failure has its roots in distant past but it still holds today with languages and tools too complex and unreliable. Languages and tools providing proper abstraction, really hiding the system beneath, would give us the simple and rigorous ground to make programs easy to prove correct, so that no testing would be needed. Unfortunately, this looks far from happening

Programming languages are further from being mathematically nice than they were 50 years ago! They’re huge and complicated. They contain big libraries, and most of a programmer’s time is spent finding and learning the right libraries.

and again

What progress has this field actually made? We still struggle with the same problems as 50 years ago: iteration times, debugging, scratching our heads trying to figure out what went wrong.

How insightful! Empirical evidence that computer science made no sizable improvement in software construction is everywhere. It’s like a hamster in a wheel, running nowhere. Why is that? Maybe, we’ve been piling leaky abstractions on leaky abstractions, apparently hiding information without really simplifying, to the point where progress is drowning in complexity.

Testing is a nice way to easily lay down executable specifications, yet it requires maintenance and quickly degrades as we try to cover more cases. But when we code we have the chance to write self-describing programs which are executable specifications, reducing the coverage of tests needed, ideally to zero.

What does it mean in practice? Whenever it’s possible one should aim at declarative code. Domain specific languages and functional programming come to mind. The point being, if a program matches closely its specification, what’s left to test?

def factorial(n)
    if n == 0
        1
    else
        n * factorial(n-1)
    end
end

Code Katas: Programmer’s Deep Practice

Wednesday, September 2nd, 2009

karate_champI’ve recently blogged about talent and how it’s grown through disciplined, committed, error focused practice at the edge of your ability, known as deep practice. I guess now it makes sense to approach it from the perspective of programming: What’s programmer’s deep practice? Unsurprisingly, inspiration can be found in the great japanese culture, people who highly value discipline and self improvement. Specifically i’m talking of martial arts. If you were to learn, say, karate you would go to a dojo and perform katas. If you happen to be a programmer, you can go to a coding dojo and practice code katas.

Code katas, a term first coined by Pragmatic Programmer Dave Thomas, are small programming exercises geared to hone a specific programming skill. Traditionally, they tend to be algorithmic like parsing or visiting graphs but could as well aim to improve understanding of particular programming paradigms, like functional or object oriented, or a specific language. Also, as remarkably pointed out by Matteo, katas can be crafted to master a certain technology like web or database. While, as you may guess, Coding Dojos are sites, groups or communities which propose and maintain collections of katas hopefully with solutions and reviews.

So, how do you practice? I suggest you solve a kata, review your work, compare it to other solutions, share your code with others and discuss it. Then solve it again trying to take a different path, balance pros and cons, then solve it again and again, until you feel you internalized the essence of the problem. Finally, you can move to another kata. If it feels like a lot of work, then you got it right. No question mastership requires time and effort but, then again, masters are those destined for greatness.

Resources