April 4, 2008

Tracking Tomatoes

I’m spending again some time to improve my time tracking process. The last step I’ve implemented is a feature I was missing before: it’s good to have the number of tomatoes spent on a given task, but how about when that tomato was consumed? This is a nice to know for me to understand the amount of context switching and why there are “holes” during the day when I’m not working tomatoes.

The idea is simple: when I assign a tomato to a task, I’d like to assign a timestamp and an optional description. I can then parse those timestamps and render some useful information on a report. It’s very important that registering the tomato, the timestamp and description is done as fast as possible so I don’t waste too much energy for tracking. If this pre-requisite is not true, I’m spending too much on tracking than the core business and of course I don’t want that

So it turned out that the process is not really different than before. The whole infrastructure is based on OmniOutliner and QuickSilver. QuickSilver is just the interface I use because I don’t want switch countinously between applications. OmniOutliner is organized so that I have projects containing user stories at the top and tasks directly below. This is how a typical session works:

  • I first select the line corresponding to the task I’m going to work next
  • Then I start a countdown timer of 25 mins (I’m currently using this)
  • When the timer’s up, I don’t touch the keyboard and I take my 5 mins break
  • When I’m back, I reset the timer and open QuickSilver to enter the description for the previous tomato spent
  • Start the countown again…

So here’s the nice AppleScript enabling the tracking (I had to split lines with a “_”):

using terms from application “Quicksilver”
on process text tomatoDescription
tell application “OmniOutliner”

tell document “projects-planning.oo3”
if selected row exists then
set selectedRow to first selected row
set newTomatoRowId to id of (make new row at_
end of rows of selectedRow with properties_
{topic:tomatoDescription})
select row id newTomatoRowId
set value of (cell “TStamp” of row id newTomatoRowId)_
to (the (current date) as string)
set value of (cell “Act” of row id newTomatoRowId)
to (“1” as integer)
else
display dialog “Please select a row.” buttons {“OK”}
return
end if
end tell

end tell
end process text
end using terms from

To activate this script, you need to save a file (for example tomato.scpt) in the ~/Library/Application Support/QuickSilver/Actions directory and restart QuickSilver. Then you can just open QuickSilver, type ‘.’, type the description text for the task and the script will do the rest. Of course there are some customization you have to do. I’m using names of specific columns and specific types from OmniOutliner.

You can download the script and the omnioutliner template if you want.

Tomatoes and GTD

If you’re using GTD you may want to integrate the Tomato technique in your process. The values added are many and important:

  • Learning how to estimate tasks and measure the actual effort taken. Why you want to do that? Because then you can plan accordingly and put priorities on tasks based also on the estimated effort.
  • Knowing how long you estimated a task is like a game. You’ll try to be consistent with the estimate and this will help you keep your focus without distractions.
  • Rhythm: the tomato sequence and breaks make your working day sustainable
March 28, 2008

The 5th Iteration

The 5th iteration was fatal.

I’ve spent the last 2 months in a very interesting project. On one side a traditional customer used to big architectures up-front and fixed bid projects and on the other a first time agile team. Of course one problem was to sell the small increments of the application to a customer expecting just a long-term release cycle. Even if the customer was happy to see the showcase at the end of the iteration and increments of the application day-by-day we suffered from the unavoidable paradigm impedance mismatch.

Symptoms are “process smells” appearing here and there. For example “user stories” introduced to enforce an architecture that the customer perceive as important. Or the need to cut off some pairing time to develop in parallel. Or situations where the team is informed that they won’t reach the deadline and they are asked how to improve velocity. More in general I’m talking about discussions to abandon an agile practices because “they are slowing us down”. These process smells are the management failure of predicting requirements that are in fact changing over time.

This is also the situation where the role of agile coach loses its meaning because the fact of getting things done is way more important than the way the goal is achieved. I also think it’s the failure of the idea that it’s possible to sell a fixed bid project as agile when it’s also fixed scope at the same time. So I’m no more involved in the project, not because I decided, but because my frustration has been perceived as negative for the good health of the team. My plan was not really different, but I was curious to see the end of the release. Damn, I missed that and I don’t want to make the same error again.

I know James Coplien believes that “TDD done strictly from the YAGNI principle leads to an architectural meltdown around iteration three”. I don’t believe that, but I can re-phrase the same sentence to say: a fixed-bid, fixed-scope agile-project leads to the typical process smells around iteration five. Can I push this a little further? A fixed-bid, fixed-scope agile project will gradually move to a non-agile project starting from iteration five and finishing the metamorphosis around release one. Of course it will be always sold as agile.

The good news? I’m back to Rails :)

March 27, 2008

Metrics Fun

Do you have some spare time at work? Waiting for the next user story? Or maybe just thinking what’s the meaning of that last step in test-code-refactor? Why not try to run some metrics? Maybe you have some legacy code sitting in your project, maybe is what you have just written. I can think of three metrics that quickly give you a great ROI: coverage, duplication and complexity. Of course there are many more, but let’s assume it’s the first time you run metrics on the project: you’ll be surprised with minimal effort.

Working on some Ruby code recently I tried Kwala, a metrics aggregator for Ruby. Kwala is wrapping other tools to create a nice report but it’s not easy to install. It requires dependencies to be installed manually (Saikuro and Amrita). Other than that, the tool seems to work great. Playing with Kwala I was able to find a nice piece of code with a red labeled complexity of “25”. I stripped all the actual lines and I left only the conditional workflow:

def blah()
  if
    case
      when
        case
          when
           
          when
           
        end
      when
        case
          when
            
        end
      when
        
      when
        
      when
        
      when
        case
          when
           
         end
      when
        case
          when
            
        end
      when
        case
          when
            
        end
      when
        
         rescue
      when
        
      when
        
      when
        
    end
  else
    
  end
  if
    return
  else
    return
    
  end
end


The answer is no, I didn’t write the code :) But the game is simple: make that complexity disappear using refactoring. There are tons of good articles and a well written books about the topic. But here’s how:

  • Search for usages of the method and try to understand how it is used and in what circumstances
  • Now go inside the method and apply some embellishment like formatting and introducing meaningful new lines to split related part of the code
  • Try to understand what the method is doing and write inline comments if necessary. A good technique is to connect the execution of the application to a debugger and follow line by line what’s going on
  • Write unit tests that mimic usages of the method by the other parts of the application
  • Write acceptances or integration tests that operates on the application interface portion which is related with that method
  • Ok, with the Refactoring book and the Refactoring Workbook opened in front of you it’s time to start refactoring.

When the game is started, developing is easier and iterations are more fun. The secret is to introduce a visual feedback, something you can race against day by day to measure your improvements. It’s exactly the same satisfaction you have in front of a green bars while running the test suite and the same applies for the other metrics like coverage or removing code duplication. Final suggestions:

  • Automate the most important metrics as part of the build
  • Make sure nice reports from the metrics are found as part of the continuous integration report
  • Make your self questions about those numbers
  • Create an habit where you consistently improve the code quality as part of the micro-cycle
  • Make metrics results part of the iteration showcase
March 9, 2008

Fun With LinkedIn

I don’t know about you, but LinkedIn never worked for me as a professional career tool. I joined at the beginning of 2005 to see what the service was all about. It helped me maintaining a resume online and know about my old friends. The reason why I say it never took off is because I’ve never seen an human resource department using it. Maybe you had a different experience and you’re happy with it. But it was very effective in one case: creating a job oriented social network of people curious about each other careers. So if I want to answer the question “but where the hell is Matt?” maybe I can search on LinkedIn and see what his new job is all about, if any.

So supposing my attitude is the same as anybody else, I decided that the serious professional LinkedIn profile was out of place. If only my old colleagues are looking at it, well, let’s have some fun! Enjoy and don’t forget to view the full profile.

March 8, 2008

The Database Irritating Inconvenience

It’s already the second or third time in my career that I encounter problems introducing a database server for object persistence from the begininng. Unfortunately in the last project I arrived too late to stop the pervasive architecture that was taking over. The problem manifests itself as long and frustrating sessions to solve deploy problems, locally or on the continous integration server or the test environment. The symptoms are:

  • People need  to set up different credentials to access the database or there are environment contraints that force different roles
  • Developers want or are forced to use different names for their database locally
  • The need to create different database instances cloning schemas and creating related scripts (unit tests wipe out the database for each tests, acceptances do not)
  • The problem that slightly different flavors of the operating system affect the way the development database is installed and accessed
  • A new developer joining the team is forced to install a local database server and follow a complex procedure to setup it correctly based on the team settings.

Having this decision made up-front can deeply impact delivery of business values.

But is a customer constraint! 

Let’s assume that the customer wants to see a database in place or change the data in the application using the database as the main interface from the beginning. Even in this case, the requirement of the customer should not interfere with development decisions. Embedded databases are fine, file system persistence is too. Xml based persistency maybe. But you should remove as fast as you can the constraints to use a specific service from the development environment especially if this service is fragile, unreliable or hard to maintain because is network based. Do you remember the first rule of distributed computing? Don’t do distribute computing.

So How?

When it’s time to have object to persist, you just need to create an ObjectStorage object or you name it, something that can accept an object instance and make it persistent. If you’re testing when you need the storage to be in place, just mock the ObjectStorage to return the list of objects you need for your test to pass. This storage is just an object abstraction of what a database will look like. At some point in time you’ll need to give the storage a real implementation and at this point, use the best tool for the job. A database server with relative client APIs is usually overkill because it produces an external depencies on the operative system and consequentely a maintenance hassle. An embedded database is better, especially if can store in-memory. Because also files suffer from the same problem: they are external dependencies on the operative system. At some point you will need to switch the developement view of the database with a real database server. Well, of course some work needs to be done, but it’s required only for the test environment and production environement. You don’t need to do the same for the CI box or the local development workstations.

Conclusions

Resist the temptation to clone the development environment exactly like you think it should be in production. In production you don’t have mutiple development workstations, you don’t have a continous integration running and it’s the only place together with the QA environment where you really need to show the architecture the customer wants to see.

March 4, 2008

Smart Driven Development

At the beginning it was very easy: you just write your test first, ahead of the application code and you’re done. Of course it turned out not to be so easy. First of all, what is a unit? It’s a class, it’s a package or what? The easy answer is: a unit is a class, so when you start test first on something, you name the unit test class as the class you intend to test. But then there are other questions you need to ansewer before coding:

  • Do you want a top-down or a bottom-up approach?
  • If I start from the top and test are failing, can I commit to the continous integration?
  • Do I need to mock objects so that I can  test also interactions? How far the interaction should be described in the test?
  • How far I have to go wih testing, should I try for 100% coverage?

The answer once again is flexibility. There is not a common answer for all situations, there is just a smart approach based on testing principles. I want to start from this interview with “Uncle” Bob Martin. Here are his 3 definitions of test driven development:

  • A test driven developer does not write a line of production code until he has written a failing unit test.
  • A test driven developer doesn’t write more of a unit test that is sufficient to fail and waits until red bars go green before writing another one.
  • A test driven developers doesn’t write more production code that is sufficient to pass a failing unit test.

But these principles are just a starting point. Only experience can tell what else you need in your daily development.

Bottom-Up or Top-Down?

I prefer the Top-Down approach whenever possible. But I’m gradually lerning that the best place to start is where you have the best understanding of the problem.  So I’ll try to start from the top, but if it makes more sense to start from the bottom because I know what to do without too much thinking, I will create higher level testing later. In this way you can be productive while you increase your understanding of the business logic. Other approach like trying to understand the problem without coding are not as effective. There is also the risk sometimes that in order to start from the acceptance test you need to solve technical problems like how to mock external services or how the result should be produced by the system. Starting from the middle/bottom layer can bring the solution to the acceptance technical problem as a side effect.

Failing Tests

It’s really great when acceptance criteria has been specified for you. Even better if those are already executable and not plain english. But also in the case the customer can only produce plain english requirements it’s not bad at all. The reason is that acceptances are those “forcing constraints” driving you in the process of writing the first tests. Acceptances run as part of the continous build, exactly like unit tests. The problem is that an acceptance will fail until you have written all the necessary unit tests and production code to make it pass. You also want to integrate in the repository with the work of the team as fast as you can. So you don’t really want a situation where you wait until the end of a user story to commit your work. Well, the reason of the acceptances is to drive the design discovery process and the writing of the unit test for the lower layers. Of course it’s also a regression test for the business requirements you’re writing. But in this situation I don’t see a problem commenting out the acceptance test from the continous integration. The same for functional/integration testing. Sure you need to remember to activate them as soon as possible. Acceptance framework or unit test framework supporting testing “categories” are a great help in this. You just need to label the test as “functional” and exclude that category from the continous integration scripts. You will also see in the report from the build something like “X test skipped” that will remind you about the exclusions.

Mocking

For the same reason an acceptance will fail at the beginning, also testing lower layers can produce the same problem. There is a big difference though. While acceptances (written by the customer) or functional/integration tests are supposed to test the real application exactly like it will work in the production environment, you don’t have the same requirement for unit tests. In other words, you can mock interactions with other component at the interface level, leaving the implementation details for the next layer of unit testing. Yes, it’s a cycle through the upper layers of the application to the bottom. In every cycle you write a suite of tests for the current layer, mocking what has not been implemented yet. So you need to mock objects or stub results, especially your approach is top-down. The risk can be to over specify. I usually specify just types for arguments to mock method calls and sometimes I just ignore arguments. I also skip verification of mock calls when the mock is not strictly related with the subject of the unit test. You should try not to create fragile unit tests. An unit test is fragile when even small changes in the production code force you to change a lot of unit tests. This situation should happen only if you touch core business values of the application where it’s worth spending more time. But if your mocks are over specified then you aren’t only testing the core business proposition of the business logic, but also all the other structural details.

Coverage

100% coverage is not possible or really difficult to achieve. The choice you make should be once again strategic: aim at higher coverage numbers for core business functionalities and leave the rest at an acceptable level. The coverage numbers are also very related to the tool you use. Only experience can say what is the coverage number and metrics that you should try to have in your project. In my experience there is no an universal metric or number you should see. There is a range you can expect (for example I consider  good a coverage between 60-80%) and then wait iteration after iteration to see how hard is to improve. You reach your optimal coverage when after putting a consistent effort on it, the metrics hardly increase.

Conclusions

I think flexibility and the capacity to contestualize experience to the surrounding environment are key factor in the success of a project. There is no silver bullet or universal rule that can be applied. There are general principles you can apply to different situation to maximize your productivity and be more effective in the less time possible.

February 24, 2008

Once again ladies and gentlemen here’s the RhinoMocks template for ReSharper to create your mocked tests in a snap. The last time was exactly the same but for JMocks with Eclipse. This time instead, the requirements are: VisualStudio, ReSharper, NUnit and RhinoMocks as references. To install:

  • Download this file (xml export of the template)
  • In VisualStudio, ReSharper menu, choose “options”
  • Go to “Templates” - “File Templates” and click the import icon
  • Select the downloaded XML file

If you just want to have a look:

using NUnit.Framework;
using Rhino.Mocks;
using Rhino.Mocks.Constraints;

namespace $namespace$
{
    [TestFixture]
    public class $class$
    {
        private $TargetType$ _a$TargetType$;
        private $CollaboratorType$ _a$CollaboratorType$;
        private MockRepository _mock;

        [SetUp]
        public void SetUp()
        {
            _mock = new MockRepository();
            _a$CollaboratorType$ = _mock.CreateMock();
            _a$TargetType$ = new $TargetType$(_a$CollaboratorType$);
        }

        [Test]
        public void TEST_METHOD_NAME()
        {
            using (_mock.Record())
            {             
            }

            using (_mock.Playback())
            {                                     

            }
        }
    }
}

February 19, 2008

www.iwasworkingwithrails.com

Starting from December I joined a cool small company called Obtiva. Don’t take that wrong: we are small by design. I think nothing will prevent Obtiva to grow and be a medium sized company one day, but this is not the goal. Obtiva is strictly people oriented and partially profit oriented. This means that Obtiva won’t accept a new project just because it’s big or profitable, but only if the project fits with developer capabilities, hopes and objectives of the company as a whole. We have an internal feedback loop to guarantee that when we set a goal, we can also measure how far or close we are. Yes, metrics, but not about code. Obtiva is not just running an Apprentice process (I’ll talk more about this in the future, but be sure to follow the work of this guy) and training classes, Obtiva is also a consultancy shop.  And here’s why “I was working with Rails”. Too bad.

With Ruby and Rails I had for sure a lot of fun. Considering that I developed in Rails just for two months, I just scratched the surface. But it was enough to promote Ruby in my toolbox. I always had a special admiration for Ruby, starting more or less 2 years ago. What attracted me was not just the neat language, but the community. For some lucky conjuncture of events (namely some Prag programmer who likes to visit far east countries in search for a new language), Ruby had  enough momentum to attract the attention of a small but very selected portion of developers. If you are a Java developer, you’ll feel lost. I strongly suggest you to use “patterns” as the learning key from the old way of thinking to the new one. This first book will make the parallel at the architecture and deploy level, while this second is a masterpiece to understand what’s wrong with GOF patterns in Ruby and how OO concepts differ between the two languages. The plan for the future is to stay on the Ruby wagon producing some side project but I have to fight with the scarcity of time since the beginning of my new role. And thinking out loud, what is my last successful side project? I have a proven record of inconsistency in the last few years. Well let’s see what happen.

February 17, 2008

Search for The New Land

It’s time for a new blog engine. iWeb is very flexible but you have to pay for that flexibility when the number of posts (or pages) is high. Now it takes more than a minute to generate the entire content. Together with comment post-processing, snippets post-processing and web synchronization this is far too much for an innocent blog post. Most of the time I’m reluctant to open iWeb and other 5(!) applications. Too hard.

So enter Tumblr, the blog engine I’m using right now. I saw one of my colleague (thanks Dave) using it and I was impressed. It’s clean, fast and MAC-like! It’s also pushing the “Keep it short dude!” model, something I was clearly missing in the past blogging. So yeah, I kinda want to be more verbose, maybe speak more about personal life and stuff and be less annoying with article-like blog post which are difficult to read.