Saturday, December 31, 2005

Update: Table of Content

I have updated the Liquid Development ToC with the lastest posts.

Friday, December 30, 2005

Tests as Storytelling

If you look at tests as your main product and to code as just an implementation of the intent expressed by tests, then we start looking at tests with greater respect and attention.

Tests deserve refactoring. They should be refined and honed so that intent is expressed exactly in one point. Well, ideally.. :-)

Tests should be read out as a story, as an explanation, as examples.
But they shouldn't be boring saying the same things over and over with little variations, in the hope of catching inconsistencies by brute force.

Reading through test should be like reading through an explanation of the system, showing you how to build up layers of abstraction and sometimes going deep in a concept, exploring all its facets.

Wednesday, December 28, 2005

Intent 0.4 - Intents, Stories and Facts

I have been keeping myself busy around christmas time.  I have corrected quite a few points of Intent and Proze, I have added a couple of examples, extended the functionalities and improved the layout.

You can download Intent-0.4 from RubyForge.

Once you have downloaded the package and made it available in ruby's load path you can run some of the samples in the demo directory.

I have already detailed intent_of_mathematics in another post, but it's worth to point out a couple of changes.
For a start I have added support for Facts.  A fact is a lazy developer's way of writing test.  You get some code that works and you literally save its context and its results, allowing you to freeze the implicit intent of this code.

This is not really intent-first development, but it is still a step towards capturing intent.

The following code records a fact about divisions:

fact!("Result of Division"){
    given "area",   a = 100
    given "height", h = 5 
    follows "base", a/h 

the fact get saved under the "facts" subdirectory as Result_of_Division.fact.xml. The xml looks like this:

<fact id='Result of Division'>

I tried to make it readable and understandable on its own, so that even a different developer will be able to understand the intent of that fact.

If the question mark of the fact call gets changed to a question mark (or simply removed)

fact("Result of Division"){ ...

then we switch from recording the fact to verifying that the fact is still true.

When I run a reality check on the intent (F5 in SciTe from the intent file) I get the following report:

---------- Reality checking Adding One ----------
Consistent intent: ..1..
Consistent intent: ..2..
Consistent intent: ..3..
---------- Reality checking Commutative Property of Addition ---
Consistent intent: ..3..
Consistent intent: ..5..

which means that the early intents are ok.

---------- Reality checking Comparison - Lesser Than ----------
Consistent intent: ..true..
**BROKEN** intent
    expected     ->false<-
    but got      ->true<-

    see: intent_of_mathematics.rb:14

which points out a broken intent and allows you to jump direcly to the offending intent code:

intent_of("Comparison - Lesser Than") {
    expect 0 < 1
    expect 1 > 2
Our division fact returns the following:

---------- Reality checking Result of Division ----------
Consistent intent: ..'area: 100'..
Consistent intent: ..'height: 5'..
Consistent intent: ..'base: 20'..

we see that our saved fact has not been violated.

The multiplication unfortunately is broken:

---------- Reality checking Result of Multiplication --
Consistent intent: ..'base: 20'..
Consistent intent: ..'height: 5'..
**BROKEN** intent
    expected     ->'area: 101'<-
    but got      ->'area: 100'<-

    see: intent_of_mathematics.rb:46

as we can see the area is wrong..

------------ Prozing ------------
prozing intent_of_mathematics.rb ...

Then the intent gets prozed and produces a readable prozed file:

require 'intent/all'

intent of "Adding One" 
     0 + 1  should be 1
     1 + 1  should be 2
     2 + 1  should be 3


intent of "Recognising Illegal Operations"
    expect exception ZeroDivisionError
        when trying to calculate
        5 / 0
story "Calculating the Area of a Triangle"
    base   = 10
    height = 5
    area   = base * height / 2
    expect 25, as area


fact "Result of Multiplication"
    given "base",   b = 20
    given "height", h = 5 
    follows "area", b*h + 1

fact "Result of Division"
    given "area",   a = 100
    given "height", h = 5 
    follows "base", a/h 

Intents are supposed to capture small grained intentions as a design device.

Stories are descriptions of typical cross-functionality scenarios.

Facts are frozen representations of implicit intent.

In the next release of Intent I will also add Snapshots, an even rougher way of capturing intents.  A snapshot will intercept and memorize a set of input and output streams (puts and gets) and will save them as unqualified facts.

What I would like to do is to create a full development path that goes from raw implicit-intent snapshots to full fledged small granularity intents.

I envision the following sequence:

snapshot --> fact --> story --> intent

I was forgetting the final report.   I'll adjust it a bit so that it fits on the web page:

Broken Expectations
 * intent: Comparison - Lesser Than - #2: expected false, but was true
 * fact: Result of Multiplication - #3: expected area: 101, but was area: 100

Broken Intents
 * Comparison - Lesser Than

Broken Facts
 * Result of Multiplication

we also get a nice report for each category and an average of expectations per intent (a good proxy of intent granularity and expressiveness)

Reality Checking
    intents: 4                stories: 2            
passed intents: 3/4     passed stories: 2/2  
broken intents: 1       broken stories: 0     
    facts: 2               expectations: 17
passed facts  : 1/2    passed expectations: 15/17
broken facts  : 1      broken expectations: 2
expectations per intent: 2.3
    powered by >>Intent<<

Tuesday, December 27, 2005

More Behaviour Driven Design - Victor Cosby and Stacy Curl's work

I have just found out Victor Cosby's Dreaming of Electric Sheep where he gives examples of Dave Astel's Behaviour Driven Design ruby framework (rSpec), providing some examples. The rSpec framework goes for the junit-like approach where every class has an associated 'test' class (Spec here). This offers some advantages over the approach that I used in Intent. For example you can introspect the Spec classes to gather some information on the name and number of specs that you are going to check.

I adopt a more free-flowing incremental kind of style, but many of my 'test' information are built only while the checks are executed.

I have also seen some work done by Stacy Curl on a unit test framework that uses PicoUnit to define tests in a pretty clean way.

There are quite a few things that I like about Stacy's work. For a start, while all the action happens within the context of tests, the code is shaped in such a way that it clearly expresses intentionality by using a reader-oriented declarative style. I also like Stacy's approach to Test evolution. You can see how the test evolves becoming richer and richer via intermediate steps that still make sense.

Finally Stacy also starts reasoning about tests in terms of an 'imaginary language'. I find this to be very forward thinking, as our basis for shaping knowledge should be based in language rather than specific programming languages. This allows us to push the boundaries of our programming languages to handle more complex situations that could be easily described in human-oriented languages.

While I use dependency injection, I don't use mocks the way Stacy does and this is probably the main difference with my work on intents. I don't usually care about the interaction of my objects with other objects and I only care for direct results. His mocks seem to check for the behaviour of side effects and this doesn't fit in my intent model well.. well, it's food for thought and it might open new perspectives for me.

Monday, December 19, 2005

A Composition for Two Audiences

One of the insights that I gained by trying to look at code with a fresh eye is that it can be useful to look at code as a communication between you and two very different kind of audiences.  The first obvious audience is the computer, the operating system and the framework you are talking to.  The second audience was first noticed by Djiekstra in his work on literate programming and rediscovered recently by the community.  The second audience is made out of people.

Your fellow developers are this audience, your clients can be this audience and, most definitely, you are this audience in the first place.

We have always implicitly acknowledged that we have to read back our own programs, even while we write them and this pushed us to create higher level languages that chunked together patterns of lower level operations.  However, as this would have been enough to tackle the task at hand in an effective way, this was not enough to communicate effectively the intent of the code to other developers or even to yourself after a few weeks without having looked at the code.

That's when developers started using subtitles.  You had bits of English interleaved with chunks of machine-friendly code.  While this gave us some indications on where we were in the code-city this was not very useful to draw paths across the code and relate different sections of code together.

The use of comments could be augmented by following a number of conventions and rules that disciplined the code back to more manageable chunks of human understandable meaning.  Meaningful names for variables.  Advanced variable declarations.  Elimination of gotos in favour of structured programming, and so on.

In this past few years, with extreme programming and ruby scrambling the rules of the game, we have come to appreciate expressive programming.  The code says what it means, so that the code can also act as documentation for itself.  We have seen this trend starting with java docs and embedded auto-generated documentation, evolving to the self-explanatory code of xp and the domain-specific-language practices of ruby.

As we have moved away from the early ideas of communicating via comments and we have embraced the idea that code should be written in such a way that it communicates to people, we have found ourselves shaping code that must please two audiences.

How easy is that to do?  If you think it's easy I challenge you to this little "boomerang translation" game.  Go on google tools for languages or on any other multilingual translation engine.  Write a simple sentence in your native language and then translate it to one or more target languages.  Now get those resulting translations and translate them back to your native language.   How much meaning has been lost?  How much of it has been distorted?  Now, the challenge is to re-write the original sentence in such a way that it still makes sense in your original language *and* it survives the double translation keeping its meaning intact as well.

Trust me this is no easy task.  You will find yourself twisting the language so that you can pack different shades of meaning in a single sentence, so that at least one will be preserved.  You will come up with conventions such as brackets, hyphens or slashes to chunk together similar meanings.

For example, if you want to express the simple concept of "This afternoon I will catch the bus to go home", you will probably need to come up with something like:

"Today Afternoon I want to (catch / use / take) the ( bus / public transportation ) to go home"

You needed to disambiguate the word "will" that in this case declined the verb to the future, but that an automated interpreter could translate as "willpower".  Also "catching" is kind of ambiguous since you don't really "catch" a bus, you just walk in it.  To avoid further ambiguities you specify a whole range of meanings "(catch / use / take)".  You would also soon realize that the word 'bus' can also have the further meaning of conduit and if you want to avoid translations such as "Here afternoon I deliberation to grab a conduit to go home" you will have to painstakingly check, disambiguate and provide alternatives for every little piece of meaning.

You will also need to go through several translation iterations to decide what works and what doesn't. 

That's a good approximation of how it feels to write for two audiences, one made of humans and the other consisting of an automated unforgiving interpreter.

Striving for a Fresh Eye

When you have been writing code for many years the difference between what is complex code and what is simple code for your fellow human beings kind of blurs away as you get used to your own ways of writing code.

It is at this stage that to make further advancements in the art of shaping knowledge you have to try to develop a perceptual stance that allows you to look at code with a fresh innocent eye, as if you saw code for the first time. Mind you, this is not easy at all as you can see something as basic and obvious, whereas a real uninitiated would only see a mumbo jumbo of code.

If you are very good at it, you will also be able to switch your perceptions between the 'fresh eye' and the 'adept eye', looking at the same code from two different perspectives. This will often allow you to see how the 'adept' can contribute to aspects of code composition that are hard for the 'novice', in a way that is natural for the novice to understand.

Just one small note of warning. Beware of showing your results to other adepts, as it is likely that they will be firmly set in their own ways and pre-judgments. Use a true novice as your benchmark and appreciate his feedback. If he doesn't get it it's probably not his fault, but a failure of yours in understanding his perceptual stance. That doesn't mean your work is useless either, but it probably isn't useful to that context and you will have to look if there is a more fitting niche where it can grow and prosper.

The Social Evolution of Software

Today I came across CodeTree.

Can digital artists learn new techniques, be exposed to new coding structures, and better express themselves by working in tandem or in a group?

CodeTree is an attempt to create a worthwhile dialogue between new media artists of different skill levels and backgrounds. The project's objective is to offer a social network that facilitates learning and artistic expression—a place where coders can dissect, share, and expand upon one another's code.

The site is nice, but what really hits me is the possibility of creating some kind of 'live' repository for code.
Ok, I know we have sourceforge, rubyforge and all those other forges and they do have forums and tools to comment on dovelopers, but I'm thinking about something different.  Smaller piecies of code for a start.  Nuggets of code that can be understood at a glance and modified with a quick brush on the keyboard. Think Java Almanac.

I would like these piecies of code to be tracked during their evolution, and to be able to follow the personal history of their contributors too.  I would like to comment on them via a wiki-like interaction.  I would love to use them via intellisense in my code. 

I want to be able to sketch my intent and as soon as my 'Intentisense' picks up interesting keywords, somethink looking a bit like Google Suggest.  If I do my own modifications I would then like to upload them directly to my Noosaurus, my shared executable knowledge thesaurus.. for other people to see and modify and rate and rank and tag.

Coding like tagging like blogging... I feel this can have huge potential.

Sunday, December 18, 2005


Intent-0.3 is out on RubyForge, and it also includes the latest version of Proze.  You can get it here.

In this version I have cleaned up things a bit and I have also included a sample intent.

Let me give you an overview of how it works.

I will assume you already have a project and some ruby files in it.  You can now write an intent file for one of your files.
Let's assume that your project is redefining mathematics and you have a mathematics.rb file.  Then you need to create an intent_of_mathematics.rb file.

Well, all you really need is to have the file to start with "intent_of", but I hope to exploit this convention in future developments to create links between code and its intents.

The first thing you do in your intent file is to import the intent definitions:

require 'intent/all'

Then you start declaring a number of intents about what your mathematical code should do.  In this example I am just parroting ruby's arithmetics.

intent_of("Adding One") {

intent_of("Commutative Property of Addition") {

Then you flesh out those intents by specifying concrete expectations:

intent_of("Adding One") {
    (0 + 1).should_be 1
    (1 + 1).should_be 2
    (2 + 1).should_be 3

intent_of("Commutative Property of Addition") {
    (2 + 1).should_be_equal_to(1 + 2)
    (2 + 3).should_be_equal_to(3 + 2)

You can even simply state that you expect something to be true or false:

intent_of("Comparison - Lesser Than") {
    expect 0 < 1
    expect 1 < 2
    dont_expect 2 < 1

You can even specify that in certain instances you expect an exception to be raised:

intent_of("Recognising Illegal Operations"){
        #when trying to calculate
        5 / 0

I made the comment a bit verbose for the purposes of Proze.

Proze is a program used by Intent to produce readable documents from the intents.

Finally you can specify Stories or Use Cases, that describe the cumulative effects of a number of operations, usually as seen by the user as a unit-sized interaction.

story("Calculating the Area of a Triangle"){
    base   = 10
    height = 5
    area   = base * height / 2
    expect 25, as=area

Now we can run a Reality Check on our expectations by simply running the program.  You just press F5 on most IDEs.
What you get is:

---------- Reality checking Adding One ----------
Consistent intent: ..1..
Consistent intent: ..2..
Consistent intent: ..3..
---------- Reality checking Commutative Property of Addition ----------
Consistent intent: ..3..
Consistent intent: ..5..
---------- Reality checking Comparison - Lesser Than ----------
Consistent intent: ..true..
Consistent intent: ..true..
Consistent intent: ..false..
---------- Reality checking Recognising Illegal Operations ----------
Consistent intent: ..'expected exception'..
---------- Reality checking Calculating the Area of a Triangle ----------
Consistent intent: ..25..
---------- Reality checking Calculating the Volume of a Prism ----------
Consistent intent: ..500..

Broken Expectations

Reality Checking
intents: 6                 expectations: 11
passed intents: 6/6        passed expectations: 11/11
broken intents: 0/6        broken expectations: 0/11
expectations per intent: 1.8

We get information on every intent, story and expectation that we have run, also producing a summary detailing the broken intents and expectations and the ratio between expectations and intents.  This ratio gives you a feeling for the granularity of your intents.

You can also create a reality-check file that can handle several intent files.  You can write a reality-check.rb file (another convention) as:

require 'intent/all'

reality_check 'intent/demo'

intent/demo is the directory containing my intent file.  All intent files in that directory get reality checked and prozed.

A prozed file gets massaged to make it more legible.  The previously described intent file becomes: prose_of_intent_of_mathematics.txt

require 'intent/all'

intent of "Adding One" 
     0 + 1  should be 1
     1 + 1  should be 2
     2 + 1  should be 3

intent of "Commutative Property of Addition" 
     2 + 1  should be equal to 1 + 2
     2 + 3  should be equal to 3 + 2

intent of "Comparison - Lesser Than" 
    expect 0 < 1
    expect 1 < 2
    dont expect 2 < 1

intent of "Recognising Illegal Operations"
    expect exception ZeroDivisionError
        when trying to calculate
        5 / 0  

story "Calculating the Area of a Triangle"
    base   = 10
    height = 5
    area   = base * height / 2
    expect 25, as area

story "Calculating the Volume of a Prism"
    base        = 10
    base height = 5
    height      = 20
    base area = base * base height / 2
    volume    = base area * height   
    expect 500, as volume

Wednesday, December 14, 2005

Functional Style

Elizabeth Keogh in her Traversing the Ocean of Knowledge blog, points out how all the evils of programming boil down to unwanted corruption of state. 

Yet, it is possible to program without keeping state.  You can produce a flow of function calls, where the objects just flow through as arguments within the functional streams.   However, the trick works only if your functions are side-effect free, as Eric Evans points out.  If your functions are side effect free they cannot corrupt any shared state and they will always return the same thing when given the same arguments.

I will give a simple practical example.
data_list_A ="..").load()
data_list_B ="..").load()

engine =,data_list_B)
result = engine.calculate(..,...,...)
Here we have three pieces of state needed to get the result we want.  If the example were slightly more complex, those states could get corrupted along the way by misbehaving code.  We also have temporal coupling here: those statements must be in the order I have written them.  They are coupled by that sequence and any attempt to change that order is likely to result in a program failure.

On the other hand I could write the same code in more functional style:

result = (

If you like to try an intentional style, you can wrap up all the nasty syntax into functions and go for the more expressive:

result = Engine( DataListA(".."), DataListB("..") ).calculate(..,...,...)

Monday, December 12, 2005

Intent & Proze

Early versions of  Intent and Proze are out on RubyForge.

I am developing Intent to allow the expression of intents, rather than tests, whereas Proze is use to produce prose out of those intents.

I am going to write more on this, but allow me to give you an early example.  This is an excerpt of code specifying the intent of different measures of interest rates

intent_of( "Defining Financial Yield Units of Measure" ) {
    #We are using a notation, where a number is
    #followed by its unit of measure;
    #for example:
    we_ expect 0.7, to_be_equal_to_( 70.percent )
    we_ expect 0.01, to_be_equal_to_( 100.bips )
Using Proze I can produce this kind of specification from the intents:

intent of  "Defining Financial Yield
            Units of Measure"

    We are using a notation, where a number is
    followed by its unit of measure;

    for example:
    we  expect 0.7, to be equal to   70 percent 
    we  expect 0.01, to be equal to   100 bips 
    We also use some alternative names for
    the same concepts.

    we  expect 2 bips, to be a synonymous
      of 2 bps 

    These measures can also be used
    interchangeably with one another.

    consider that:

    we should  expect 2 percent,
       to be equivalent to   200 bps 


Now let's try something slightly tougher..
let's play with present value

intent of  "Specifying Money Currency"  
    First of all, we must be able to define Money
    we  expect 3, to be the "Money(3).amount"
    we  expect "EUR", to be "Money(3).currency"
    by default
but we  expect "USD", to be
    the "Money(3,"USD").currency" 


intent of  "Using Cashflows"  
    Let's now deal with cashflows

    with  Conventional discount =
          continuously compounded yield

    we  expect about 271.67748, as the
        present value of 
                  Cashflow  [
                        100 at 1 year ,
                        100 at 2 years ,
                        100 at 3 years 
                  ]    ,

        with interest rate at 5 percent  

I don't know about you, but I can read this stuff out aloud with the expectation of being understood by domain experts.

Saturday, December 10, 2005

Liquid Development ToC

A short index to the material of this and other blogs, in reading order.

Liquid Development
Writing Styles
Shaping VBA and Excel into an Agile Platform
The Nature of the Material Inhabited Software
The Art of Growing Software
The Way of Meta
Shaping Workshop

Abstraction in Practice
Mastery Projects

Sunday, December 04, 2005

Playing Between Sand and Water

As a child, as many children do, I liked to play with sand on the beach, close to the sea.  I used to sit just a bit further away from the farthest point the longer waves would hit, and there I started to build.  I knew that, as the time passed, the odd wave would have hit my structure and that, with the passage of time, the whole structure would eventually dissolve, but I also knew that I would still have the time to see buildings, cities and civilisations evolve there, between sand and water, on the border of the sea.

I started out with big solid chunky lumps of sand, building up a solid irregular wall against the water.  In my mind those walls and those lumps of sand represented a primitive culture, just getting the jist of culture and civilisation, trying to ward off the entropic currents of the sea.

Then, on top and around those structures I started building more elaborate decorations.  The irregular buildings started getting a squared shape, and small walls started running between buildings.  This represented a young culture just discovering the value of functional shapes.

Inevitably a wave hit at one point, dissolving one wall and destroying part of my culture.  This destruction created vacuums where a new different culture or subculture could emerge and be shaped.  Depending on how it evolved and the waning and waxing of the waves, it could take over or disappear.

Baroque cultures appeared and took over the world, forgetting about the waves, building high pinnacles and towers into the sky, drawing beautiful arcs and digging tunnels close, too close to the sea… until they got destroyed almost at once by their own hubris.

More conservative cultures used most of their energies to built tall and thick walls to keep the water away.  The lack of water however didn't allow building much beyond those walls and, after a while walls and buildings just got dry and started crumbling.

Smart cultures were built around the very concept of wave.  They were shaped in such a way that the waves got channelled along canals into artificial lakes, turning the destructive force of water into a useful tool for further construction. 

However even smart cultures would become complacent and stopped building, stopped caring about new waves, sometimes coming from a different direction, sometimes of different size, and they also got eroded one bit at a time and disappeared.. leaving behind beautiful but unusable tunnels and artefacts for the newcomers.

The waves were, in my mind, the forces of time and change, the barbaric invasions and natural disasters, the forces of entropy continuously eating away at the culture.

More than twenty years later, while contemplating a kid playing on the beach of Antibes on the Cotes Azure, all these thoughts came back to my mind and I was surprised realising how close they were to development, the continuous evolution of structures, layers over layers of civilisation, always under siege by both the external and internal forces of entropy, always ready to fall to hubris and complacency.

And I realized with joy that I never stopped doing what I liked and what I wanted, I never really stopped playing with sand.

This page is powered by Blogger. Isn't yours?