Reblog

Building Highly Available Systems in Erlang

Joe Armstrong gave an interesting and entertaining talk about the principles behind Erlang. The original slides are available here and the talk will be soon online on the InfoQ website. Especially enjoyed a few bits from his past working at satellites software and Ericsson. Here’s my summary, enjoy.

High availability (or HA) is quite difficult to achieve in practice. 10 nines availability, or 99.99999999% uptime is not that far away in the future and Erlang is designed just to achieve that goal. HA is very close to our everyday life than you expect: the washing machine runs HA software, pace makers, aircraft control systems of course. But even if the effort made is huge, there is always the possibility a cosmic ray hitting the circuit board ad flipping a couple of bits.

http://www.erlang-solutions.com

Something that wasn’t a problem a while ago is internet HA, where “internet” means networked applications. We want to take advantage of this possibility, that is, to have data redundancy for HA on multiple nodes and the computation to happen in any of those nodes. For a washing machine for example, data and computation are in the same place. Availability increases with multiple distributed copies of the same data because it lowers the probability that all data copies are corrupted at the same time.

If we bring the data redundancy strategy to the extreme using as many as possible nodes, there are other problems to solve. Here’s a possible scenario: a system with 10 millions “cores” (machines or CPUs) and 10 copies of the data. How to find those data? The Chord address protocol solves this issue. An md5 of the ip address of all machines is created and sorted. The key id of the stored data is hashed as well and that value is then compared with the values of the hashed IPs. That copy of the data will be stored on the closest node by lexical ordering. A replica can be stored by hashing again the key as many times as I want. The probability of 5 copies of the data failing at the same time is 10 square -10, that is 0.00000000001 or our ten nines availability. Data HA problem solved. But what about the availability of the computation?

The key enablers for HA computation can be summarised in six laws.

Rule #1: isolation

A failure in a module should not affect other modules. Sometimes a dependent module hides a circular dependency you should be aware of. For example if I put my data on both Dropbox and S3 I should be aware that if S3 goes down also Dropbox that depends on it goes down. Two modules of an application, one based on Dropbox and the other on pure S3 are then not isolated.

Rule #2: concurrency

The world is not sequential, is embarrassingly parallel. Modelling such a world with imperative languages is difficult. Also consider that we need a system with at least two computers to make it a non-stop system.

Rule #3: must detect failures

A defect cannot be fixed if the system doesn’t know anything about it. For example, to determine if it was the machine or the communication channel that died. It implies distributed error handling, since at the time you collect data about the error those data must be handled by a non-faulty machine.

Rule #4: failure identification

Implies that you need to collect all required information about the failure in order to do something about it.

Rule #5: live code upgrade

Of course there is no point in stopping a server to fix it when what you’re trying to achieve is HA. The AXE-10 telephone exchange system HA requirement was no more than 10 mins downtime per year.

Rule #6: stable storage

When the system stores data, it is forever. Data is distributed accordingly to avoid data corruption. There shouldn’t be any external backup required.

Just to give some more context, the 6 laws of high available systems aren’t completely new ideas. You can find analogies in old papers, for example Jim Gray from Tandem computers in “Why computer stops and what should be done about it” of 1985. Erlang is not that different than a Tandem architecture written in software. The general principle is that system should be designed for failure and not completely in the direction of preventing failure. For example, I don’t bring with me a defribillator because of a potential heart attack in the middle of the street and I don’t try open heart surgery on myself. What I care is that a team of well instructed doctors can arrive on time equipped with the right tooling to save my life. Emergency doctors are trained to deal with human failure despite a great deal of effort is also put on prevention. Another important takeaway from the paper is to address the failure as fast as possible because the more you wait the more it gets worse.

Software modularity should be achieved through isolation and messages. Messages are very important at the point Alan Key created SmallTalk, a language based on message passing between objects. But unfortunately the part for which SmallTalk is so best known are Objects and Classes. Alan Key was frustrated by this, as he reported on the Squeak mailing list

The 6 laws can be applied as a library or as part of the language itself. A library cannot be that effective if the language doesn’t support the 6 laws natively. If for example rule number #1 “isolation” is implemented in a language that only allows an application to run inside a single process then it becomes programmer responsaibility to orchestrate multiple processes. Of course Erlang was built with these principles in mind. Let’s review Erlang features accordingly to the six rules.

Rule #1: Erlang processes are isolated and cheap. It takes 360k of memory for a new process. It’s more complicated to handle failure when pieces of the same application share same memory space. We are expecting a lot more of miniaturization, up to 1000 cores on a single FPGA which means around 100 millions processes all running on the same chip.

Rule #2: Erlang processes are concurrent. Message passing between processes is concurrent. Every process comes with a mailbox that can be inspected regoularly, but the check doesn’t stop the process from doing other things.

Rule #3: Erlang processes handle failure. The mailbox is not used for error propagation, signals are used. A process can catch the signal and do something about it. If the signal is not trapped the monitoring process will exit as well. At this point becomes essential to incorporate as much possible information for the trapping process to handle the error.

Rule #4: A classic problem in parallel computing is how to stop a cluster of running processes when something goes wrong. This is why you want some of them to be special error handlers using the trap signal mechanism. The Erlang OTP provides sensible defaults to handle errors. Riak, CouchDB, Facebook (the Erlang based chat) all uses mechanism like this one.

Rule #5: Thanks to modules Erlang can be upgraded as it runs. When you invoke a function the module it runs in is by default the last version. If you deploy another version of that module, the current code will starting running the old version unless it is preferred otherwise. When Erlang was invented memory was expensive, so at the beginning only a last version and an old version were available at the same time. Nowadays you can select amonsgst different versions.

Rule #6: Mnesia is the database included with Erlang OTP. That is the primary Erlang offer for data that persist forever. Or you can use one of the many good quality 3rd parties (CouchDB, Riak).

What’s beautiful about enforcing all the six rules is that you get an incidental HA system as a result. Erlang is looking at the HA problem considering failure just a special case of success. Concurrent computation becomes like finding islands of stability and move from one stable state to the other as the system progresses.

The Data Panorama

Gosh, it has been a while since the last post. The following is the summary of the opening keynote Martin Fowler and Rebecca Parson gave at this year QCon London. I re-phrased most of the sentences but the sequence of thoughts and main ideas should be intact. Enjoy.

The nature of data is changing over time and with that the way we process it. One thing over all is that data is becoming “Big Data”. By definition data is big when the size of the data itself becomes the problem. For example the challenges that Facebook with its 40 million photos or Walmart with 1 million transactions per hour are facing.

Of course also the technology available today is different. Today we prefer to use grids of standard commodity hardware instead of big mainframes and data is becoming distributed. Distributed data means that also your washing machine today contains valuable data, along with cars, cellphones and other standard appliances. The growth of networked sensor nodes is about 30% every year. Distributed data is managed in a distributed way as well: Wikipedia receives something like 1.5 million contributions per month with an average of 10 edits each contributors.

Data is also changing its nature. It’s now valuable in the sense that is “urgent”: the faster it is accessed the more value it acquires. Think about social journalism. The sense of how much is urgent depends on the type of business processing it: it can be valuable for one second, one day, one week. Often it is more valuable when different source of information are interconnected. Data interconnection 20 years ago was described as “data silos” and constituted a poor solution to the problem of creating relationships in data. Today we prefer to create a centralised data model that create interconnections between silos often modeled as an “enterprise data model”. Today we also know that this model is flawed and creates coupling and rigidity.

As data evolved also solutions to process them evolved as well. Scaling the enterprise model horizontally doesn’t work that good and it creates a rigid structure of data that cannot easily be changed. What we allow nowadays is a certain overlap between the compartmentalised data source (billing systems, orders, etc) and a way to translate between a model into another. The best centralized solution to aggregate and manage these differences is NoSql. NoSql is non-relational allowing to relax the schema and aggregate similar data with slightly different structures.

There are also different flavours of NoSql systems. Document based systems are great for aggregates that are then usually used for analysis. The document based nosql database knows how to deal with differences in data. But sometimes the aggregate gets in the way. When slicing and dicing is everything you do with data, graph databases are a better solution. Other simpler ways of storing data in an unstructured way are key-value stores. You can mix these types of nosql databases together for different requirements. Let’s call it polyglot persistence.

New application styles and architectures also emerged. In event based systems a “change” is transformed into an object instead of being directly processed. An “engine” is then responsible of processing those events. Events processing is logged so it’s always possible to rebuild history in case of some kind of catastrophic failure. Cloud topologies seem to be the best fit to store this new kind of inherently delocalized data. The Cloud also allows for processing on demand keeping the costs down enabling analysis on demand.

Data analysis evolved as well. Data warehouses are now less optimal because of the new nature of data which tend to be schema-free and data has become not only text but also video, audio, images. Roll-ups, trends and variances are more difficult to extract because of the data “fluidity”. Also map-reduce kind of analysis are less effective if you don’t know exactly what to ask. Perhaps data mining techniques like pattern recognitions will become more valuable to deal with data fluidity.

The way we consume data is also different. Instead of extracting known trends we “search” data for unexpected patterns. The role that used to be more the one of a “data scientist” is now better specified as “data journalisms”. Data journalists know how to turn data into a story that everybody can read. For example: the google search box on one side and the iGoogle interface on the other. In the first case the user knows what to search for. In the second case the tool tries to infer what information is “good” for the user. A good book describing what data journalism means is Agile Analytics by Ken Collier. Data ubiquity is also responsible for crowd sourced data analysis. A few example: http://ureport.ug/, ushahudi platform at http://ushahidi.com/products/ushahidi-platform, Kenya open data initiative http://opendata.go.ke/, data without borders http://datawithoutborders.cc

As data becomes more accessible the problem of privacy should be also addressed, as well as the problem of altering the information for malicious goals. Quoting HAL 9000 from a popular scene of 2001 Space Odyssey: “A series 9000 HAL computer never made any error or currupted any data”.

RSpec, minitest, cukes & friends

I followed the discussion on Twitter after David Heinemeier Hansson declared his dislike for rspec and similar testing frameworks. This is nothing new but this time he was pissed off by some code snippet he was working on. I’ve read several interesting answers and the general consensus seems to be that there is not an absolute “right” way to spec-out features. Agree.

I always used RSpec (except for an RR parenthesis I wasn’t happy about) but these days I’m more inclined to go the simplest route, that is for Rails the included Test::Unit and integration test capabilities. Also outside Rails I’m more inclined to use Test::Unit/minitest which comes for free with Ruby 1.9. What is pushing me away from RSpec is the lack of consistency in the DSL release after release.

New keywords are introduced constantly which most of the time are equivalent ways to express the same construct. I think this is David Chelimsky tying to integrate all the good suggestions coming from the community. Although I appreciate his effort, the final result for me is that I struggle to read specs written by someone else unless I learn the new syntax. The other problem is that working on different projects with different RSpec versions I often can’t remember what style I can use. Embellishments to RSpec aren’t for sure a bad thing except when the DSL becomes a moving target and the changes introduced can’t be considered definitive “improvements” but different ways to name the same thing. I’d rather prefer David to be much more opinionated about RSpec, for the good and for the bad.

I have the impression that with Cucumber, Capybara, RSpec and the incredible number of other frameworks we are pushing the need for testing way too far. It’s like we “rubysts” are trying to demonstrate where we can push a DSL for testing thanks to the language expressiveness, which is an useful not business-related exercise. There are of course use cases for all of the above frameworks, but the community effort in the direction of testing frameworks seems disproportionate.

Quick Getting Started with Clojure

Here is a quick list of opinionated links on how to get started with Clojure I prepared for a friend recently. I think it adds some value on top of a standard Google search, especially for beginners, by excluding a few (but very good indeed) advanced resources you can discover later.

• The crash video introduction. If you have a Java background, have a look at the standard Rich Hickey intro part 1 and part 2 intro from blipTV. If you have a Ruby background you could enjoy also the Stuart Halloway’s introduction for rubyists. Having seen both and having both Java and Ruby experience, I suggest you to cope with the Java syntax if you can and watch Rich Hickey first. Stuart tends to compress concepts (both here and in his book, Programming Clojure by the pragmatic).
• If you don’t like watching videos (I like them a lot, but I can’t easily find the right time window to do it) you can start right away writing code with the Clojure Koans. Of all the possible assisted tutorials, the koans format has the best “zero to keyboard” acceleration. In all cases, try not mess around with the Java classpath at this point if you can. In case you feel disoriented about how to install and run the github koans on your computer, read this good koans preparation article, but skip Leiningen and just place the jar suggested from the Github page into the lib directory.
• If you want an offline option, I suggest Practical Clojure by Luke Vanderhart and Stuart Sierra over the older and more practical Programming Clojure by Stuart Halloway. The introductory material in Practical Clojure is very well crafted, including a gentle introduction to the functional programming paradigm shift and a mind-opening explanation of the meaning of state in object orientation. Stuart’s book is for sure another option but also a more advanced step in my opinion, being essential and practical.
• Fully equipped with your introductory material I suggest you to attend a programming dojo. A dojo is a form of collective pair programming with very simple rules to rotate people at the keyboard. The dojo forces you to put the new knowledge into practice and at the same time you’ll get in contact with more experienced Clojure programmers full of expertise to share and make you faster and better. London Clojurians for example are building a simple game with Clojure as a learning tool and meetup once a month. Hopefully you’ll find a dojo close to you, or maybe you can create your own!

How to find ideas for a talk

Here is a quick list of actions to consider the next time you run out of ideas while thinking at the next conference to speak at. I consistently run through the list and usually come out with a fair list of topics. Next step is of course to select the best ones and create the abstract. Each bullet in the list can take a little while to execute properly. Half a day is necessary if you didn’t use the list for a while, but following times should be much faster:

• What you did during the past year? What did you learn that is worth mentioning in a talk?
• If you keep a daily log (I do) look at notes about problems you fixed or things you’ve learned. Anything interesting in there?
• Have a look at other “trendy” conferences in the close future and see what the buzz is all about. Have you worked with the same technology/process stack? Do you think you know enough about it to present yourself?
• Have a look at the past conferences you attended to see if there are topics worth presenting somewhere else and spreading the word
• Have a look at the last past talks you gave (if any) to see if some of them are improvable and still actual
• Have a look at the last article/podcasts/books you bookmarked to see if those topics are worth speaking about
• Have a look at past talks held at the same conference (the one you’d like to submit to) to understand their preferences and skim the topic list

Hope this helps

Ivy Lee and the evolution of modern time management

Just recently I learned about Ivy Lee (not the actress) who is considered one of the first public relations specialist in history. Ivy Lee is considered also the “inventor” of a simple yet powerful focus management technique branded “top 5” in our modern times by Cameron Herold.

The technique can be summarized as follow:

• At the end of the current working day write down the six most important things you should do next day
• Start from the top of the list first thing next working day and stick with it until done
• Start executing the next task only after you finish with the current one
• At the end of the day reconsider priorities and create a new task list

The short sized list forces you to consider what really matters and leave everything else out. It turns out that your main priorities follow the Pareto Principle and you should end up doing 80% of what really matters with those top priorities. What I just told you is the short explanation but please read the more “romantic” full story about how Bethlehem Steel Corporation was able to become one of the largest steel producer in the US thanks to Ivy Lee advices.

Top5 includes a lot of what is used in modern time management techniques like the Pomodoro Technique or GTD. The Pomodoro Technique focuses on sequential execution of tasks extracted from a prioritized todo-today list which only contains tasks worth the capacity planned for that day. GTD contains (amongst the rest) the notion of current/tomorrow (and then monthly. quarterly and life-time) most important achievements. Both GTD and the PT then expands the concept in different ways and directions making it a complete (and complicated) framework.

The simplicity of Top5 is only apparent: it requires a fair amount of self-discipline to list the top priorities and execute them, leaving a lot of room for misinterpretation and mistakes. Teamly.com, a website I discovered recently, gives its Top5 interpretation adding a few features to help you out.

For example a today’s priority can’t be edited after 20 minutes from creation to force you really consider if this is a top priority or not. Of course you can’t create more than 5 but you can create other priorities for the next day, next week or month. In general an user of the technique is delegated the following decisions:

• How big a priority should be
• How to prioritize them
• How to maintain focus on the current task (despite internal or external interruptions)
• How to measure effectiveness in tasks execution

and possibly more. My impression is that Top5 is too easy to execute incorrectly “as is” bringing to early failure. Top5 has very few prescriptions and hence is more an expert tool and a good candidate for integration with other time management techniques.

SC2010 Report

I attended the Software Craftmanship 2010 conference in Bletchley Park and had a great time. A craftsmanship based conference is a collective exercise in clean programming using a lot of different techniques: katas, koans and maybe less esoteric exercises in refactoring and test driven development. This was exactly what I expected and the reason for me to attend. I coded the whole time and concentrated on my craft, tooling and techniques. Here are the sessions I attended.

Refuctoring with Jason Gorman

I briefly looked at the screencast of the presentation the day before and thought refuctoring must be fun and it was. The technique is simple: grab a very simple line of code, like “Hello World” which is fully covered with tests and try to make it a mess! Not easy at all! There are so many ways you can be a bad programmer and we are so used to try hard for maintainability that writing crappy code is excruciatingly difficult :)

But refuctoring is not just fun. It is a real exercise with good overall effects. Since you’re not concerned with maintainability of the code or how to better structure business rules, the focus is just purely on refactoring with all the benefits of recognizing quickly how to clean up the mess you just created. Kudos to Jason Gorman for the whole conference and for his very entertaining session. I know now how to recognize code “smiles” and replacing them with code “smells”.

Designer Code by Chris Parsons

This was the natural continuations of the previous session. After a brief introduction to code smells and related refactoring moves, Chris Parsons of Eden Development fame showed us a completely cryptic piece of code to refactor, probably the work of a Master Refuctor. Code coverage was already in place for the exercise, we just had to solve the puzzle of defining meaningful name and clean up the code. Another very good exercise I couldn’t complete but hopefully is all available here.

Make Mine A Mocker by Neill Pearman

Another interesting way of exercising. This time, Neill created for us a koan based on a cucumber feature set on the path of building a mocking library for Ruby. Following the steps and the tips given, we were required to write enough code to make the scenario pass and move to the next failing ones. I couldn’t get through the end (as almost all the other exercise) but it’s easy to “doggy bag” the content of the conference thanks to GitHub and bring back the koan with you for later enlightenment.

Code Breaker History

After lunch we got the tour of Bletchley Park and of course learn about the important piece of cryptography history that tool place here during world war 2. Alan Touring was head of the first team who architected the Bomb, an electromechanical calculator to exploit the settings of the Enigma machine. What a nice way to spend some time outside and relax before the afternoon sessions!

Robot Tournament by Matt Wynne

I really enjoyed the development challenge organized for us by Matt Wynne. Very simple: create a small program able to play tic tac toe in a tournament with 6 “battle” and hopefully win. I preferred Ruby for my robot but I wasn’t that good :( I managed to win 50% of the battles until the 4th round, then trying to add the new feature I ran out of time. Another useful exercise: adding the time constraint to the mix forces you to think about the best possible solution in the least amount of time. Of course the solution cannot be the best and the focus is exactly this. Release quickly release often is the key to survive in such an environment and developing this kind of challenge you’re forced to think about compromise. Kudos to Matt for his tournament platform available here. I will certainly suggest the game at the next craftsmen meetup.

This week’s highlights

Show #80 Josh Clark, Tapworthy: Designing Great iPhone Apps - The Startup Success Podcast… Josh Clark is not telling us the usual commonplace stuff we use to hear when speaking about what makes a mobile application great. The big picture consists of three mindsets: micro-tasking (repetition), locality (urgency), boredom. I’m on the go when I use my iPhone so I’m micro-tasking to get something done quickly. The application should allow me to get to the final results in just a few quick moves. I’m local, so I expect from the application to localize information and to mix “local” with sensors: visualize what is local, speak to me what is local or tell me what’s in front of me thanks to the compass. We can give new context to the “personal computer” definition of decades ago. Speaking of porting legacy app to mobile, Josh suggests not to just replicate the experience on the phone, but to find what part can be really be mobile. Mobile platforms aren’t just for work, they are cultural platforms, something with a personality.

Show #74 Kent Beck on Lean Startups, TDD and startups The Startup Success Podcast… The “Learn, Measure, Build Cycle” is at the base of continuous deployment. You can think of it as an A/B Testing which takes place ahead of feature completion. The principle is to build just enough software just to get the answer to the business question. Then deploy and measure the results. Capital requirements and risk will go down as a consequence. Capital efficiency for startups has increased to the point that a single person can work to more startup projects at once. VCs of course enjoy this because they can increase the number of startups to fund at a lower cost and increase the probability of success of at least one. Sometimes it’s necessary to take a shortcut and avoid testing. TDD is especially effective for the not well known areas of the application and for parts subject to heavy optimization. The decision to skip testing is not a practice you can take in isolation, it’s an option when it considerably slows down the dev cycle.

The Clean Coder: The Hacker, The Novice, The Artist, and The Craftsman.… Never thought of that in this terms, but the ROI-based model Uncle Bob describes covers a lot of cases. The craftsman is the only category described here that cares about ROI. Wonder where researchers would fit in here. There is definitely some sort of ROI in research, but the time to achieve ROI is different that that of the industry.

Dan North - Deliberate Discovery… Dan North speaks on his blog about what we are missing when we just concentrate on user stories during the inception of a project with the effect of having stakeholders looking at the story backlog, turning it 90 degrees and think of it as a gaant chart. Deliberate discovery means to allocate part of the inception effort to something which is not story description. Learning is the constraint. Ignorance decreases as the project moves forward in a non linear fashion. The skill to improve is how to find which factors in a project (or ignorance axis) are the most effective to reduce ignorance the quickest and take whatever action is needed in that direction along with user story planning. Dan is convinced that this skill is subject to the Dreyfuss model and as such it requires practice and this is why we have the impression we can’t predict what’s going to happen to the project. Of course we don’t have the crystal ball, but we can get better at it.

Herding Code 82: Cory Foy and Will Green Compare .NET and Ruby Development | Herding Code… Comparisons between languages are usually useless or even stupid. I heard a very discussion instead in this PodCast by Herding Code thanks to Cory Foy. First of all ActiveRecord and its SRP violation: in Ruby with mixins and dynamic types it’s much easier to add persistency into model objects. This is probably the main reason why Rails developers usually don’t care. With typed languages instead the amount of code and interfaces required to achieve the same thing is evident and scarier. “Your application tends to model the company organizational structure” so people think that Java apps are more complicated because they model complex businesses. False: Ruby is just so expressive that you don’t need a lot of layers just to make it works. DDD is uncommon because Rails came out from a presentation first approach that somehow permeates into the framework.

Episode 165: NoSQL and MongoDB with Dwight Merriman | Software Engineering Radio… Another nice episode on the nosql db space. I think I’m getting closer and closer to understand what kind of problem nosql dbs are solving. One of the reasons is that the relational model is very easy to understand and intuitive: the transactional ACID properties is exactly what we want to see. I write something and I can read my changes. I move data from one place to another and I expect to see the new data in the right place. And so on. NoSql is just opening to all the shade of grey in between. Data will be eventually consistent is the key: concurrency and availability is embraced. Programmers now have one more task each user story developed: what is the consistency model? Based on the answer, different part of the application can be tuned for different consistency. For example if I am the user making the change I want to see my changes, but I don’t care if others will see in a few minutes. There are many other nuances like this one which are not easy with MySql.

Twitter / Francesco Cirillo: …right now 5 minute… This tweet from Francesco, the inventor of the Pomodoro Technique, remembers me that the 25 mins pomodoro “time” is customizable. The fact that the timer can be adjusted during the day is probably one of the most important aspect of the technique. The pomodoro time should contract or expand to fit the current focus ability. If you can’t focus for more than 5 minutes, then that should be your pomodoro time. The Pomodoro time can be be reverted slowly up to 25 minutes or more when internal distractions aren’t an issue anymore. Pomodoro shrinking accomplishes a specific goal: keep frustration away when we are not able to concentrate and keep on working with satisfaction even when bad mood kicks in.

Gojko Adzic - Thought-provoking TDD exercise at the Software Craftsmanship conference… This article describe a constrained mode TDD where the implementation that makes the test passes is first written directly inside the test, hence without structural plumbing like methods or classes. The implementation moves to classes only with a set of restricted steps. I consider this a good exercise to test everyday assumptions while applying TDD, like right jump to create a class to make the test pass. The need for structural containment is discovered through testing exactly like the implementation details. As someone noticed in the comments, this style leads to a functional style of programming since the first encapsulation of code admitted are methods.

Why: A Tale Of A Post-Modern Genius - Smashing Magazine… A nice write-up about the _Why persona, who he was and what he did in computer programming and arts. If ever had the need to send someone who doesn’t know _why reading something about him, this is for sure a well written biography. _Why is another example (the other being Alan Kay) of person who was inspired by young people for their programming activities, treating them like “customers”. There is definitely something to learn looking at kids while they try to force a system to do what they want. Their naivety is almost impossible to replicate in our adult world but it is fundamental in interaction design. So if you have a kid, don’t miss the opportunity to watch him/her closely while they learn because it’s an invaluable source of inspiration.

iPad Usability: First Findings From User Testing (Jakob Nielsen’s Alertbox)… Hard to believe, but the iPad’s bigger screen poses serious challenges to interaction designers. I’d be tempted to say it’s just a bigger iPhone, no big deal but that’s not the case. On the iPad the text that is big enough to read is not big enough to tap, for example. Another problem that the bigger screen seems to aggravate is the lack of a strong touch interface standard and users get confused on what to expect when tapping. About swiping: should you swipe bottom to top or right to left to scroll? The screen format doesn’t help to discover what on the iphone is evident. Personally I found the same issues on the iPhone as well. For example, a great application like Stanza always confuses me when I try to reach the homepage with the book index. I always end up on the topic list for the current book.

Better Software Conference 2010 Part 1

I just attended the 2010 edition of Better Software conference in Florence. Overall it was a good conference with probably the best italian speakers around. The only problem for “Better Software” was that there was not enough software. Sure, many interesting talks and a lot of great informal chat during breaks. But there was very little code on the giant screens and way too many pictures. Many attendees in the room were indeed coding while watching the talk (there was also a good 50% twitter users typing though). Others were programming outside at the lobby. I hope to see more code the next year, perhaps with a specific track dedicated to languages, programming, tools and techniques. Code examples and live sessions are the most important learning tools for programmers in my opinion and managers should know how to code too! :)

End of the (mild) rant, now the good part. There are tons of smart programmers and entrepreneurs in Italy. I had the opportunity to meet many at the conference and I was inspired in several ways. I met also a lot of young programmers eager to learn and improve their skills. Sometimes you just need to stay away from institutionalized environment (where I used to work) to discover enthusiasm and energy and Better Software succeeded attracting exactly this kind of people. Good job.

Here’s a summary of the talk I attended and some thoughts they provoked. Don’t miss the conference next year, I won’t.

Stefano Sanna - Application store, potenzialità e trappole

Although Symbian sold 80M+ devices, apps created for Symbian OS are not selling that well. With just 2.5M of devices Apple is instead the best seller with 4 billion downloads and 185k published apps. Of course developers won’t invest in a platform that doesn’t sell applications. The conclusion is that the device spread is not enough to succeed if there isn’t a store capable of selling applications. Other interesting fact is that the best sellers are those (like Apple) who sell devices with a mandatory data contract. Without a data plan you can’t install apps directly from the device which is easier than synchronizing from a desktop. Speaking of the two main contenders, AppStore and Android, a surprise comes from the number of downloads per device which is roughly similar for the two platforms. But this is a potentially misleading metric because Android applications can be purchased anywhere, not just from the official store. Many applications sold outside the official channel are thus un-trackeable.

Omar Cafini - Sviluppare e vendere applicazioni per iPhone

Omar is an interesting mix between technical capabilities and entrepreneurship. He runs a couple of events: MarcheIN and 24hrsCamp that remembers me similar initiatives I found in the States. He’s also a programmer and with a few friends he’s running an iPhone development firm, mitapp, that contains already a well rounded application portfolio. Here’s a few points from his presentation about the experience of developing iPhone applications.

• Games software houses have big set of titles on the store, which is difficult to compete against as an indie developer. Unfortunately this means that quantity is sometimes more important than quality in the Apple Store.
• Prefer the team to the single developer more or less for the same reasons as above.
• Think twice before sending the app to the appstore. The visibility effect during the first moments of life is almost non-repetible. In other words the top of visibility is reached at the very beginning.
• The market is saturated with apps at the 0.99$ price target. Again another warning that is difficult to make a living when you’re forced to sell at such a low price.
• Some interesting marketing techniques help to achieve some results if carefully tailored before launching: in app purchase, redeem codes or contest based on redeem codes. Do not forget to reserve a domain and a supporting web site for the app.

Something I also experienced first person that Omar was talking about is the kind of users using iPhone apps. Don’t forget that the iPhone is now more a fashion object than a real instrument for mobile workers. Sad but true, be “stupid” designing applications and contents, because this is the kind of public you’re aiming at. Please be merciful if you ever see me creating a fart app! :)

Leandro Agrò - Una storia di SW dai protocolli alla startup

Unfortunately I wasn’t able to understand what the talk was about. I had reminiscences of Jini (the Java technology) at some point, but I’m not sure. I saw images, videos and heard craploads of buzzwords. I know they are creating (or created) an usb crocodile called iCrocco who remembers “feelings” that can be uploaded to the desktop when connected. Or maybe not. Who knows? Please enlighten me.

Andrea Santagata - Start-up web in Italia. Si può fare

This is the man behind several italian popular websites and services like Splinder. The talk was an objective take on the italian internet market as seen from the startups he is responsible for. Some of them succeeded some of them failed. I really appreciated he was so open about both, a rare example of honesty and sustainable business (he was concerned about employees families). The last “creature” is called Liquida and as far as I can tell it resembles WolphramAlpha for the italian market. Andrea is still waiting to call Liquida a success. At the end of the presentation he just nailed the biggest problem for the success of an internet startup: all citizens should be able to access a DSL connection. In other words Internet should be a government priority and be part of our rights. Needless to say, we are not driving the effort in Italy, quite the opposite.

It is taking longer than expected to re-organize my notes about the conference. I decided to split my report into multiple parts. I’m working on part 2, stay connected. Cheers.

This week’s highlights

REWORK: The new business book from 37signals.… The book by 37signals is not saying a lot of new stuff. But it’s not a surprise. I’m an avid reader of their blog and followers of podcasts and presentations by DHH, Jason Fried and friends and they talk constantly about what was condensed in this book. Nevertheless this is a good reading. First of all is not easy to express what creates a profitable company in a succinct way like they did. Second, I like the format: small illustrated chapters that you can read easily. Of course there was also something to learn. Reading the history and motivation behind the success of whatever human endeavor gives a lot of insight for current projects and the book is full of great examples. Favorites: focus on what won’t change, underdo your competition, four letter words.

markupslicer | Slice your HTML/CSS markup into Ruby on Rails templates… Just a great idea. This service allows you to automate that repetitive action of splitting a web page into functional areas that are then worked separately. This includes also css tags, a time waster every time I need to think a name for a section. I can see it especially useful to extract partials for a Rails application but other frameworks will be supported soon.

Three Rivers Institute  » Blog Archive  » Continuous Deployment Immersion… Nice. This should be called Extreme Deployment and be part of the Extreme Programming practices set. What Kent describes here gives you a method to move the build process to continuous deployment step by step the hard way. Why? Search my other links, there are a few talking about lean startup thinking and when continuous deployment makes a lot of sense: essentially to move the product more close to what the customer wants instead of what you think they need.

What will happen when a software company downs tools for a week? - Business of Software Blog… Here’s an expanded version of what Relevance is doing every Friday. With a week available to create something which doesn’t need to be related with your company current stream of revenue, what will you do? I have plenty of ideas and the fact that your company supports you is nice. But as someone said in the comments, it should probably be expanded to whatever you have in mind, company related or not, open or closed source. In my opinion it is a win-win even if the whole company contributes to open source projects for a week. It’s an huge return of image, free advertisement and word of mouth and not only. What was contributed outside the company will return in as side thinking, new knowledge and ideas. Well done anyway, although it should sounds more like a practice for the long term instead of something to try once a year. Hope they’ll consider this for the time being.

Chariot Tech Cast - Battle of the Frameworks II… An interesting debate. I don’t usually like panels because they are too broad and unfocused. This is an exception but the panel is not interesting for the web framework discussion itself (which is just a little more than a marketing pitch) but for the several unrelated topics in between. JSF is a hostile abstraction web framework because it brings too far away from HTML in a unnatural way. “Successful languages like Objective-C or JavaScript are really abominations”, while framework like JQuery are there to fix ten years old problems of browsers and make it a modern language. CSS flexible boxes, dom local storage and other HTML 5 tricks are not yet organically explained in tutorials to build up complete applications. The TIOBE index is bullshit because it reflects the inability of search engine to return coherent results when the language name is not enough (like Go for example or Prototype that requires an additional “programming” term). And much more, enjoy.

Hacker News | HN Contractors: Add Yourself… A resource of valuable information, like rates and what people are up to for their contracting activity, with emails and web sites. This list helped me understand how to adjust my actual rate and also gave me an opportunity to look at web sites of people I value like the HN community.