May 1, 2008

dTunes First Release

Ladies and gentlemen, a first absolutely-beta release of “Disposable Tunes” is now available online at http://dtunes.reborg.net. It’s probably not more than a prototype, but it demonstrate what can be done. So here’s a possible start for a FAQ page:

What’s dTunes?
dTunes is an mp3 fuzzy search engine and player. It mashes together results from Yahoo! search, Yahoo! Pipes and Feedity. It’s fuzzy because it’s based on those public accessible directories that come and go at any time in the web. So the results displayed right now might not be available in the next five minutes. This is the reason of the “D” at the beginning of the name: media in dTunes are disposed by design.

How can I use it?
In it’s current form without playlists, you open the dTunes web page ans start a search. If results are available, you can start playing them right away. The dTunes page becomes your “desktop radio” for a couple of hours or the day. Then you close it without the need of knowing how to save, open or find those songs again.

How dTunes differs from, say, SeeqPod.com or the tons of websites with similar capabilities?
There is one big difference: there is no server side. Your browser is hosting the application completely and is talking with other services on the web to grab the information it needs. For you as an end user this is not a big deal. But from the maintenance and scalability point of view can become important (assuming that dTunes will be used by millions of people in the future).

What browsers are supported?
Currently Firefox, Safari and Camino with Flash support.

What are the missing features?
Playlist is probably the missing feature right now. Playlists enable you to decide the sequence of songs to play, so you can do it once and enjoy a couple of hours of music. Technically there are many enhancements to be done. The list of matching sources can be expanded using other search engines or asking for more results from the same search engine. A “buy” option can also be a nice to have: since you know that the song you really like is going to “expire” one day or another, buying it is a good option to store it permanently. And many many more.

Well, let me know what you think.

April 19, 2008

The Elastic Server

I’ve been impressed by the presentation I attended this afternoon at the Chicago Ruby Group about the Elastic Servers. Maybe there are already other services like this but I didn’t know about it before.

You can go to this web site and have a VMWare, Parallels, Xen or others virtualization tool server prepared for you in minutes. After you download the image you can install your applications on the virtual server and re-package the whole thing for distribution. But there is more. The site offers you the possibility to deploy the server with your customization to one of the cloud services available: Amazon EC2, Google AppEngine and many more. Your image can also be made publicly available for others to download.

Virtualization farms aren’t a new concept. The difference here is that you don’t need to be a system admin to create a VMWare image to distribute with your applications on it. The operative system is installed and configured with the package you want and an admin web tool is deployed also so that you can start, stop, install applications on the virtual server without any knowledge of unix services.

Here’s are some cool scenarios where you can use virtualized servers on demand:

  • Demo to the customer: you can create the server, install your apps on it and invite the customer to download and run the image. She can play with the application in a completely isolated environment that you have prepared
  • Start of a new project: all the necessary infrastructure to collaborate with the customer can be deployed on an Elastic Server. One for each project! On the virtual server you can put the wiki, the mailing list, the tracking application, spreadsheets, project management application, continuous integration server and the source control system. You need to create then some customization script to give the correct names and procedure everywhere. You can deploy this server to some external cloud service, so neither you nor the customer need to allocate hardware for this.
  • New developers on the team: this is especially good for consultancy situation. No more reading a huge list of tasks on the wiki for the new developer. You just need to prepare an image with the development environment ready to go, one for each project.
I’m sure there are many many more scenarios for virtualization. The only limit I see to this online service is that you can install only free operative systems on the Elastic Server. So if you need a Windows environment for some reason, you have to make your own.
April 18, 2008

Stop Making your Own Agile!

I heard many times people talking about the fact that they are agile because they don’t strictly follow “the practices”. Good, I think. Then they describe that their TDD approach is like: first they write down some code, then they test and because testing make them think about possible corner cases, then they expand the first core with additional logic. Here’s another example about pairing: since I am in a room with 3 other people, when I’m stuck I ask one of them to help me understand what’s going wrong with my code. Another style of pairing is instead the situation where a team try to pair up when the pressure on the project is not too tight: a degenerated collective ownership.

I think that Beck’s 2nd book has been largely misunderstood. There is a reason to introduce flexibility applying a process and the possibility for a team to change practices or introduce new ones. But those changes are the product of an hard work of the team made of metrics and continuous retrospective. You start from the practices and when you realize that something is wrong in a deterministic way (i.e. numbers), then you change the practices.

Being agile doesn’t mean that you’re so smart that in front of a set of rules you are empowered to skip or change those rules. Being agile means that after you followed the rules or best practices and collected a feedback (or you contextualized to a specific environment), you have the power to change those rules and adapt the process for the team. I know that what I said before sounds naive (with all that good literature around about craftsmanship, journeyman to master and pragmatism) but it looks like that the temptation to skip the hard work and jump to the master status became equivalent to agility these days.
April 4, 2008

Tracking Tomatoes

I’m spending again some time to improve my time tracking process. The last step I’ve implemented is a feature I was missing before: it’s good to have the number of tomatoes spent on a given task, but how about when that tomato was consumed? This is a nice to know for me to understand the amount of context switching and why there are “holes” during the day when I’m not working tomatoes.

The idea is simple: when I assign a tomato to a task, I’d like to assign a timestamp and an optional description. I can then parse those timestamps and render some useful information on a report. It’s very important that registering the tomato, the timestamp and description is done as fast as possible so I don’t waste too much energy for tracking. If this pre-requisite is not true, I’m spending too much on tracking than the core business and of course I don’t want that

So it turned out that the process is not really different than before. The whole infrastructure is based on OmniOutliner and QuickSilver. QuickSilver is just the interface I use because I don’t want switch countinously between applications. OmniOutliner is organized so that I have projects containing user stories at the top and tasks directly below. This is how a typical session works:

  • I first select the line corresponding to the task I’m going to work next
  • Then I start a countdown timer of 25 mins (I’m currently using this)
  • When the timer’s up, I don’t touch the keyboard and I take my 5 mins break
  • When I’m back, I reset the timer and open QuickSilver to enter the description for the previous tomato spent
  • Start the countown again…

So here’s the nice AppleScript enabling the tracking (I had to split lines with a “_”):

using terms from application “Quicksilver”
on process text tomatoDescription
tell application “OmniOutliner”

tell document “projects-planning.oo3”
if selected row exists then
set selectedRow to first selected row
set newTomatoRowId to id of (make new row at_
end of rows of selectedRow with properties_
{topic:tomatoDescription})
select row id newTomatoRowId
set value of (cell “TStamp” of row id newTomatoRowId)_
to (the (current date) as string)
set value of (cell “Act” of row id newTomatoRowId)
to (“1” as integer)
else
display dialog “Please select a row.” buttons {“OK”}
return
end if
end tell

end tell
end process text
end using terms from

To activate this script, you need to save a file (for example tomato.scpt) in the ~/Library/Application Support/QuickSilver/Actions directory and restart QuickSilver. Then you can just open QuickSilver, type ‘.’, type the description text for the task and the script will do the rest. Of course there are some customization you have to do. I’m using names of specific columns and specific types from OmniOutliner.

You can download the script and the omnioutliner template if you want.

Tomatoes and GTD

If you’re using GTD you may want to integrate the Tomato technique in your process. The values added are many and important:

  • Learning how to estimate tasks and measure the actual effort taken. Why you want to do that? Because then you can plan accordingly and put priorities on tasks based also on the estimated effort.
  • Knowing how long you estimated a task is like a game. You’ll try to be consistent with the estimate and this will help you keep your focus without distractions.
  • Rhythm: the tomato sequence and breaks make your working day sustainable
March 28, 2008

The 5th Iteration

The 5th iteration was fatal.

I’ve spent the last 2 months in a very interesting project. On one side a traditional customer used to big architectures up-front and fixed bid projects and on the other a first time agile team. Of course one problem was to sell the small increments of the application to a customer expecting just a long-term release cycle. Even if the customer was happy to see the showcase at the end of the iteration and increments of the application day-by-day we suffered from the unavoidable paradigm impedance mismatch.

Symptoms are “process smells” appearing here and there. For example “user stories” introduced to enforce an architecture that the customer perceive as important. Or the need to cut off some pairing time to develop in parallel. Or situations where the team is informed that they won’t reach the deadline and they are asked how to improve velocity. More in general I’m talking about discussions to abandon an agile practices because “they are slowing us down”. These process smells are the management failure of predicting requirements that are in fact changing over time.

This is also the situation where the role of agile coach loses its meaning because the fact of getting things done is way more important than the way the goal is achieved. I also think it’s the failure of the idea that it’s possible to sell a fixed bid project as agile when it’s also fixed scope at the same time. So I’m no more involved in the project, not because I decided, but because my frustration has been perceived as negative for the good health of the team. My plan was not really different, but I was curious to see the end of the release. Damn, I missed that and I don’t want to make the same error again.

I know James Coplien believes that “TDD done strictly from the YAGNI principle leads to an architectural meltdown around iteration three”. I don’t believe that, but I can re-phrase the same sentence to say: a fixed-bid, fixed-scope agile-project leads to the typical process smells around iteration five. Can I push this a little further? A fixed-bid, fixed-scope agile project will gradually move to a non-agile project starting from iteration five and finishing the metamorphosis around release one. Of course it will be always sold as agile.

The good news? I’m back to Rails :)

March 27, 2008

Metrics Fun

Do you have some spare time at work? Waiting for the next user story? Or maybe just thinking what’s the meaning of that last step in test-code-refactor? Why not try to run some metrics? Maybe you have some legacy code sitting in your project, maybe is what you have just written. I can think of three metrics that quickly give you a great ROI: coverage, duplication and complexity. Of course there are many more, but let’s assume it’s the first time you run metrics on the project: you’ll be surprised with minimal effort.

Working on some Ruby code recently I tried Kwala, a metrics aggregator for Ruby. Kwala is wrapping other tools to create a nice report but it’s not easy to install. It requires dependencies to be installed manually (Saikuro and Amrita). Other than that, the tool seems to work great. Playing with Kwala I was able to find a nice piece of code with a red labeled complexity of “25”. I stripped all the actual lines and I left only the conditional workflow:

def blah()
  if
    case
      when
        case
          when
           
          when
           
        end
      when
        case
          when
            
        end
      when
        
      when
        
      when
        
      when
        case
          when
           
         end
      when
        case
          when
            
        end
      when
        case
          when
            
        end
      when
        
         rescue
      when
        
      when
        
      when
        
    end
  else
    
  end
  if
    return
  else
    return
    
  end
end


The answer is no, I didn’t write the code :) But the game is simple: make that complexity disappear using refactoring. There are tons of good articles and a well written books about the topic. But here’s how:

  • Search for usages of the method and try to understand how it is used and in what circumstances
  • Now go inside the method and apply some embellishment like formatting and introducing meaningful new lines to split related part of the code
  • Try to understand what the method is doing and write inline comments if necessary. A good technique is to connect the execution of the application to a debugger and follow line by line what’s going on
  • Write unit tests that mimic usages of the method by the other parts of the application
  • Write acceptances or integration tests that operates on the application interface portion which is related with that method
  • Ok, with the Refactoring book and the Refactoring Workbook opened in front of you it’s time to start refactoring.

When the game is started, developing is easier and iterations are more fun. The secret is to introduce a visual feedback, something you can race against day by day to measure your improvements. It’s exactly the same satisfaction you have in front of a green bars while running the test suite and the same applies for the other metrics like coverage or removing code duplication. Final suggestions:

  • Automate the most important metrics as part of the build
  • Make sure nice reports from the metrics are found as part of the continuous integration report
  • Make your self questions about those numbers
  • Create an habit where you consistently improve the code quality as part of the micro-cycle
  • Make metrics results part of the iteration showcase
March 9, 2008

Fun With LinkedIn

I don’t know about you, but LinkedIn never worked for me as a professional career tool. I joined at the beginning of 2005 to see what the service was all about. It helped me maintaining a resume online and know about my old friends. The reason why I say it never took off is because I’ve never seen an human resource department using it. Maybe you had a different experience and you’re happy with it. But it was very effective in one case: creating a job oriented social network of people curious about each other careers. So if I want to answer the question “but where the hell is Matt?” maybe I can search on LinkedIn and see what his new job is all about, if any.

So supposing my attitude is the same as anybody else, I decided that the serious professional LinkedIn profile was out of place. If only my old colleagues are looking at it, well, let’s have some fun! Enjoy and don’t forget to view the full profile.

March 8, 2008

The Database Irritating Inconvenience

It’s already the second or third time in my career that I encounter problems introducing a database server for object persistence from the begininng. Unfortunately in the last project I arrived too late to stop the pervasive architecture that was taking over. The problem manifests itself as long and frustrating sessions to solve deploy problems, locally or on the continous integration server or the test environment. The symptoms are:

  • People need  to set up different credentials to access the database or there are environment contraints that force different roles
  • Developers want or are forced to use different names for their database locally
  • The need to create different database instances cloning schemas and creating related scripts (unit tests wipe out the database for each tests, acceptances do not)
  • The problem that slightly different flavors of the operating system affect the way the development database is installed and accessed
  • A new developer joining the team is forced to install a local database server and follow a complex procedure to setup it correctly based on the team settings.

Having this decision made up-front can deeply impact delivery of business values.

But is a customer constraint! 

Let’s assume that the customer wants to see a database in place or change the data in the application using the database as the main interface from the beginning. Even in this case, the requirement of the customer should not interfere with development decisions. Embedded databases are fine, file system persistence is too. Xml based persistency maybe. But you should remove as fast as you can the constraints to use a specific service from the development environment especially if this service is fragile, unreliable or hard to maintain because is network based. Do you remember the first rule of distributed computing? Don’t do distribute computing.

So How?

When it’s time to have object to persist, you just need to create an ObjectStorage object or you name it, something that can accept an object instance and make it persistent. If you’re testing when you need the storage to be in place, just mock the ObjectStorage to return the list of objects you need for your test to pass. This storage is just an object abstraction of what a database will look like. At some point in time you’ll need to give the storage a real implementation and at this point, use the best tool for the job. A database server with relative client APIs is usually overkill because it produces an external depencies on the operative system and consequentely a maintenance hassle. An embedded database is better, especially if can store in-memory. Because also files suffer from the same problem: they are external dependencies on the operative system. At some point you will need to switch the developement view of the database with a real database server. Well, of course some work needs to be done, but it’s required only for the test environment and production environement. You don’t need to do the same for the CI box or the local development workstations.

Conclusions

Resist the temptation to clone the development environment exactly like you think it should be in production. In production you don’t have mutiple development workstations, you don’t have a continous integration running and it’s the only place together with the QA environment where you really need to show the architecture the customer wants to see.

March 4, 2008

Smart Driven Development

At the beginning it was very easy: you just write your test first, ahead of the application code and you’re done. Of course it turned out not to be so easy. First of all, what is a unit? It’s a class, it’s a package or what? The easy answer is: a unit is a class, so when you start test first on something, you name the unit test class as the class you intend to test. But then there are other questions you need to ansewer before coding:

  • Do you want a top-down or a bottom-up approach?
  • If I start from the top and test are failing, can I commit to the continous integration?
  • Do I need to mock objects so that I can  test also interactions? How far the interaction should be described in the test?
  • How far I have to go wih testing, should I try for 100% coverage?

The answer once again is flexibility. There is not a common answer for all situations, there is just a smart approach based on testing principles. I want to start from this interview with “Uncle” Bob Martin. Here are his 3 definitions of test driven development:

  • A test driven developer does not write a line of production code until he has written a failing unit test.
  • A test driven developer doesn’t write more of a unit test that is sufficient to fail and waits until red bars go green before writing another one.
  • A test driven developers doesn’t write more production code that is sufficient to pass a failing unit test.

But these principles are just a starting point. Only experience can tell what else you need in your daily development.

Bottom-Up or Top-Down?

I prefer the Top-Down approach whenever possible. But I’m gradually lerning that the best place to start is where you have the best understanding of the problem.  So I’ll try to start from the top, but if it makes more sense to start from the bottom because I know what to do without too much thinking, I will create higher level testing later. In this way you can be productive while you increase your understanding of the business logic. Other approach like trying to understand the problem without coding are not as effective. There is also the risk sometimes that in order to start from the acceptance test you need to solve technical problems like how to mock external services or how the result should be produced by the system. Starting from the middle/bottom layer can bring the solution to the acceptance technical problem as a side effect.

Failing Tests

It’s really great when acceptance criteria has been specified for you. Even better if those are already executable and not plain english. But also in the case the customer can only produce plain english requirements it’s not bad at all. The reason is that acceptances are those “forcing constraints” driving you in the process of writing the first tests. Acceptances run as part of the continous build, exactly like unit tests. The problem is that an acceptance will fail until you have written all the necessary unit tests and production code to make it pass. You also want to integrate in the repository with the work of the team as fast as you can. So you don’t really want a situation where you wait until the end of a user story to commit your work. Well, the reason of the acceptances is to drive the design discovery process and the writing of the unit test for the lower layers. Of course it’s also a regression test for the business requirements you’re writing. But in this situation I don’t see a problem commenting out the acceptance test from the continous integration. The same for functional/integration testing. Sure you need to remember to activate them as soon as possible. Acceptance framework or unit test framework supporting testing “categories” are a great help in this. You just need to label the test as “functional” and exclude that category from the continous integration scripts. You will also see in the report from the build something like “X test skipped” that will remind you about the exclusions.

Mocking

For the same reason an acceptance will fail at the beginning, also testing lower layers can produce the same problem. There is a big difference though. While acceptances (written by the customer) or functional/integration tests are supposed to test the real application exactly like it will work in the production environment, you don’t have the same requirement for unit tests. In other words, you can mock interactions with other component at the interface level, leaving the implementation details for the next layer of unit testing. Yes, it’s a cycle through the upper layers of the application to the bottom. In every cycle you write a suite of tests for the current layer, mocking what has not been implemented yet. So you need to mock objects or stub results, especially your approach is top-down. The risk can be to over specify. I usually specify just types for arguments to mock method calls and sometimes I just ignore arguments. I also skip verification of mock calls when the mock is not strictly related with the subject of the unit test. You should try not to create fragile unit tests. An unit test is fragile when even small changes in the production code force you to change a lot of unit tests. This situation should happen only if you touch core business values of the application where it’s worth spending more time. But if your mocks are over specified then you aren’t only testing the core business proposition of the business logic, but also all the other structural details.

Coverage

100% coverage is not possible or really difficult to achieve. The choice you make should be once again strategic: aim at higher coverage numbers for core business functionalities and leave the rest at an acceptable level. The coverage numbers are also very related to the tool you use. Only experience can say what is the coverage number and metrics that you should try to have in your project. In my experience there is no an universal metric or number you should see. There is a range you can expect (for example I consider  good a coverage between 60-80%) and then wait iteration after iteration to see how hard is to improve. You reach your optimal coverage when after putting a consistent effort on it, the metrics hardly increase.

Conclusions

I think flexibility and the capacity to contestualize experience to the surrounding environment are key factor in the success of a project. There is no silver bullet or universal rule that can be applied. There are general principles you can apply to different situation to maximize your productivity and be more effective in the less time possible.

February 24, 2008

Once again ladies and gentlemen here’s the RhinoMocks template for ReSharper to create your mocked tests in a snap. The last time was exactly the same but for JMocks with Eclipse. This time instead, the requirements are: VisualStudio, ReSharper, NUnit and RhinoMocks as references. To install:

  • Download this file (xml export of the template)
  • In VisualStudio, ReSharper menu, choose “options”
  • Go to “Templates” - “File Templates” and click the import icon
  • Select the downloaded XML file

If you just want to have a look:

using NUnit.Framework;
using Rhino.Mocks;
using Rhino.Mocks.Constraints;

namespace $namespace$
{
    [TestFixture]
    public class $class$
    {
        private $TargetType$ _a$TargetType$;
        private $CollaboratorType$ _a$CollaboratorType$;
        private MockRepository _mock;

        [SetUp]
        public void SetUp()
        {
            _mock = new MockRepository();
            _a$CollaboratorType$ = _mock.CreateMock();
            _a$TargetType$ = new $TargetType$(_a$CollaboratorType$);
        }

        [Test]
        public void TEST_METHOD_NAME()
        {
            using (_mock.Record())
            {             
            }

            using (_mock.Playback())
            {                                     

            }
        }
    }
}