Liquid Development

I have updated the Liquid Development ToC with the lastest posts.

Tests should be read out as a story, as an explanation, as examples.

I have been keeping myself busy around christmas time.  I have corrected quite a few points of Intent and Proze, I have added a couple of examples, extended the functionalities and improved the layout.

You can download Intent-0.4 from RubyForge.

Once you have downloaded the package and made it available in ruby's load path you can run some of the samples in the demo directory.

I have already detailed intent_of_mathematics in another post, but it's worth to point out a couple of changes.
For a start I have added support for Facts.  A fact is a lazy developer's way of writing test.  You get some code that works and you literally save its context and its results, allowing you to freeze the implicit intent of this code.

This is not really intent-first development, but it is still a step towards capturing intent.

The following code records a fact about divisions:

fact!("Result of Division"){
    given "area",   a = 100
    given "height", h = 5 
   
    follows "base", a/h 
}

the fact get saved under the "facts" subdirectory as Result_of_Division.fact.xml. The xml looks like this:

<fact id='Result of Division'>
    <area>100</area>
    <height>5</height>
    <base>20</base>
</fact>

I tried to make it readable and understandable on its own, so that even a different developer will be able to understand the intent of that fact.

If the question mark of the fact call gets changed to a question mark (or simply removed)

fact("Result of Division"){ ...

then we switch from recording the fact to verifying that the fact is still true.

When I run a reality check on the intent (F5 in SciTe from the intent file) I get the following report:

---------- Reality checking Adding One ----------
Consistent intent: ..1..
Consistent intent: ..2..
Consistent intent: ..3..
---------- Reality checking Commutative Property of Addition ---
Consistent intent: ..3..
Consistent intent: ..5..

which means that the early intents are ok.

---------- Reality checking Comparison - Lesser Than ----------
Consistent intent: ..true..
**BROKEN** intent
    expected     ->false<-
    but got      ->true<-

    see: intent_of_mathematics.rb:14
            D:/apps/ruby/lib/ruby/1.8/intent/demo/intent_of_mathematics.rb:14:in

which points out a broken intent and allows you to jump direcly to the offending intent code:

intent_of("Comparison - Lesser Than") {
    expect 0 < 1
    expect 1 > 2

I have also seen some work done by Stacy Curl on a unit test framework that uses PicoUnit to define tests in a pretty clean way.

One of the insights that I gained by trying to look at code with a fresh eye is that it can be useful to look at code as a communication between you and two very different kind of audiences.  The first obvious audience is the computer, the operating system and the framework you are talking to.  The second audience was first noticed by Djiekstra in his work on literate programming and rediscovered recently by the community.  The second audience is made out of people.

Your fellow developers are this audience, your clients can be this audience and, most definitely, you are this audience in the first place.

We have always implicitly acknowledged that we have to read back our own programs, even while we write them and this pushed us to create higher level languages that chunked together patterns of lower level operations.  However, as this would have been enough to tackle the task at hand in an effective way, this was not enough to communicate effectively the intent of the code to other developers or even to yourself after a few weeks without having looked at the code.

That's when developers started using subtitles.  You had bits of English interleaved with chunks of machine-friendly code.  While this gave us some indications on where we were in the code-city this was not very useful to draw paths across the code and relate different sections of code together.

The use of comments could be augmented by following a number of conventions and rules that disciplined the code back to more manageable chunks of human understandable meaning.  Meaningful names for variables.  Advanced variable declarations.  Elimination of gotos in favour of structured programming, and so on.

In this past few years, with extreme programming and ruby scrambling the rules of the game, we have come to appreciate expressive programming.  The code says what it means, so that the code can also act as documentation for itself.  We have seen this trend starting with java docs and embedded auto-generated documentation, evolving to the self-explanatory code of xp and the domain-specific-language practices of ruby.

As we have moved away from the early ideas of communicating via comments and we have embraced the idea that code should be written in such a way that it communicates to people, we have found ourselves shaping code that must please two audiences.

How easy is that to do?  If you think it's easy I challenge you to this little "boomerang translation" game.  Go on google tools for languages or on any other multilingual translation engine.  Write a simple sentence in your native language and then translate it to one or more target languages.  Now get those resulting translations and translate them back to your native language.   How much meaning has been lost?  How much of it has been distorted?  Now, the challenge is to re-write the original sentence in such a way that it still makes sense in your original language *and* it survives the double translation keeping its meaning intact as well.

Trust me this is no easy task.  You will find yourself twisting the language so that you can pack different shades of meaning in a single sentence, so that at least one will be preserved.  You will come up with conventions such as brackets, hyphens or slashes to chunk together similar meanings.

For example, if you want to express the simple concept of "This afternoon I will catch the bus to go home", you will probably need to come up with something like:

"Today Afternoon I want to (catch / use / take) the ( bus / public transportation ) to go home"

You needed to disambiguate the word "will" that in this case declined the verb to the future, but that an automated interpreter could translate as "willpower".  Also "catching" is kind of ambiguous since you don't really "catch" a bus, you just walk in it.  To avoid further ambiguities you specify a whole range of meanings "(catch / use / take)".  You would also soon realize that the word 'bus' can also have the further meaning of conduit and if you want to avoid translations such as "Here afternoon I deliberation to grab a conduit to go home" you will have to painstakingly check, disambiguate and provide alternatives for every little piece of meaning.

You will also need to go through several translation iterations to decide what works and what doesn't. 

That's a good approximation of how it feels to write for two audiences, one made of humans and the other consisting of an automated unforgiving interpreter.

Today I came across CodeTree.

Can digital artists learn new techniques, be exposed to new coding structures, and better express themselves by working in tandem or in a group?

CodeTree is an attempt to create a worthwhile dialogue between new media artists of different skill levels and backgrounds. The project's objective is to offer a social network that facilitates learning and artistic expression—a place where coders can dissect, share, and expand upon one another's code.

The site is nice, but what really hits me is the possibility of creating some kind of 'live' repository for code.
Ok, I know we have sourceforge, rubyforge and all those other forges and they do have forums and tools to comment on dovelopers, but I'm thinking about something different.  Smaller piecies of code for a start.  Nuggets of code that can be understood at a glance and modified with a quick brush on the keyboard. Think Java Almanac.

I would like these piecies of code to be tracked during their evolution, and to be able to follow the personal history of their contributors too.  I would like to comment on them via a wiki-like interaction.  I would love to use them via intellisense in my code. 

I want to be able to sketch my intent and as soon as my 'Intentisense' picks up interesting keywords, somethink looking a bit like Google Suggest.  If I do my own modifications I would then like to upload them directly to my Noosaurus, my shared executable knowledge thesaurus.. for other people to see and modify and rate and rank and tag.

Coding like tagging like blogging... I feel this can have huge potential.

Intent-0.3 is out on RubyForge, and it also includes the latest version of Proze.  You can get it here.

In this version I have cleaned up things a bit and I have also included a sample intent.

Let me give you an overview of how it works.

I will assume you already have a project and some ruby files in it.  You can now write an intent file for one of your files.
Let's assume that your project is redefining mathematics and you have a mathematics.rb file.  Then you need to create an intent_of_mathematics.rb file.

Well, all you really need is to have the file to start with "intent_of", but I hope to exploit this convention in future developments to create links between code and its intents.

The first thing you do in your intent file is to import the intent definitions:

require 'intent/all'

Then you start declaring a number of intents about what your mathematical code should do.  In this example I am just parroting ruby's arithmetics.

intent_of("Adding One") {

...

}

intent_of("Commutative Property of Addition") {

...

}

Then you flesh out those intents by specifying concrete expectations:

intent_of("Adding One") {

    (0 + 1).should_be 1

    (1 + 1).should_be 2

    (2 + 1).should_be 3

}

intent_of("Commutative Property of Addition") {

    (2 + 1).should_be_equal_to(1 + 2)

    (2 + 3).should_be_equal_to(3 + 2)

}

You can even simply state that you expect something to be true or false:

intent_of("Comparison - Lesser Than") {

    expect 0 < 1

    expect 1 < 2

    dont_expect 2 < 1

}

You can even specify that in certain instances you expect an exception to be raised:

intent_of("Recognising Illegal Operations"){

    expect_exception(ZeroDivisionError){

        #when trying to calculate

        5 / 0

    }

}

I made the comment a bit verbose for the purposes of Proze.

Proze is a program used by Intent to produce readable documents from the intents.

Finally you can specify Stories or Use Cases, that describe the cumulative effects of a number of operations, usually as seen by the user as a unit-sized interaction.

story("Calculating the Area of a Triangle"){

    base   = 10

    height = 5

    area   = base * height / 2

   

    expect 25, as=area

}

Now we can run a Reality Check on our expectations by simply running the program.  You just press F5 on most IDEs.
What you get is:

---------- Reality checking Adding One ----------
Consistent intent: ..1..
Consistent intent: ..2..
Consistent intent: ..3..
---------- Reality checking Commutative Property of Addition ----------
Consistent intent: ..3..
Consistent intent: ..5..
---------- Reality checking Comparison - Lesser Than ----------
Consistent intent: ..true..
Consistent intent: ..true..
Consistent intent: ..false..
---------- Reality checking Recognising Illegal Operations ----------
Consistent intent: ..'expected exception'..
---------- Reality checking Calculating the Area of a Triangle ----------
Consistent intent: ..25..
---------- Reality checking Calculating the Volume of a Prism ----------
Consistent intent: ..500..

---------------------------------------------------------
Broken Expectations
---------------------------------------------------------
---------------------------------------------------------

----------------
Reality Checking
---------------------------------------------------------
intents: 6                 expectations: 11
---------------------------------------------------------
passed intents: 6/6        passed expectations: 11/11
broken intents: 0/6        broken expectations: 0/11
---------------------------------------------------------
expectations per intent: 1.8
---------------------------------------------------------

We get information on every intent, story and expectation that we have run, also producing a summary detailing the broken intents and expectations and the ratio between expectations and intents.  This ratio gives you a feeling for the granularity of your intents.

You can also create a reality-check file that can handle several intent files.  You can write a reality-check.rb file (another convention) as:

require 'intent/all'

reality_check 'intent/demo'

intent/demo is the directory containing my intent file.  All intent files in that directory get reality checked and prozed.

A prozed file gets massaged to make it more legible.  The previously described intent file becomes: prose_of_intent_of_mathematics.txt

require 'intent/all'

intent of "Adding One" 

     0 + 1  should be 1

     1 + 1  should be 2

     2 + 1  should be 3

intent of "Commutative Property of Addition" 

     2 + 1  should be equal to 1 + 2

     2 + 3  should be equal to 3 + 2

intent of "Comparison - Lesser Than" 

    expect 0 < 1

    expect 1 < 2

    dont expect 2 < 1

intent of "Recognising Illegal Operations"

    expect exception ZeroDivisionError

        when trying to calculate

        5 / 0  

story "Calculating the Area of a Triangle"

    base   = 10

    height = 5

    area   = base * height / 2

   

    expect 25, as area

story "Calculating the Volume of a Prism"

    base        = 10

    base height = 5

    height      = 20

   

    base area = base * base height / 2

    volume    = base area * height   

   

    expect 500, as volume

Elizabeth Keogh in her Traversing the Ocean of Knowledge blog, points out how all the evils of programming boil down to unwanted corruption of state. 

Yet, it is possible to program without keeping state.  You can produce a flow of function calls, where the objects just flow through as arguments within the functional streams.   However, the trick works only if your functions are side-effect free, as Eric Evans points out.  If your functions are side effect free they cannot corrupt any shared state and they will always return the same thing when given the same arguments.

I will give a simple practical example.

data_list_A = DataListA.new("..").load()
data_list_B = DataListB.new("..").load()

engine = Engine.new(data_list_A,data_list_B)
engine.setup(...)
result = engine.calculate(..,...,...)

Here we have three pieces of state needed to get the result we want.  If the example were slightly more complex, those states could get corrupted along the way by misbehaving code.  We also have temporal coupling here: those statements must be in the order I have written them.  They are coupled by that sequence and any attempt to change that order is likely to result in a program failure.

On the other hand I could write the same code in more functional style:

result = Engine.new (

             DataListA("..").new.load(),

             DataListB("..").new.load()

         ).calculate(..,...,...)

If you like to try an intentional style, you can wrap up all the nasty syntax into functions and go for the more expressive:

result = Engine( DataListA(".."), DataListB("..") ).calculate(..,...,...)

Early versions of  Intent and Proze are out on RubyForge.

I am developing Intent to allow the expression of intents, rather than tests, whereas Proze is use to produce prose out of those intents.

I am going to write more on this, but allow me to give you an early example.  This is an excerpt of code specifying the intent of different measures of interest rates

intent_of( "Defining Financial Yield Units of Measure" ) {
    #We are using a notation, where a number is
    #followed by its unit of measure;
    #for example:
    we_ expect 0.7, to_be_equal_to_( 70.percent )
    we_ expect 0.01, to_be_equal_to_( 100.bips )

Using Proze I can produce this kind of specification from the intents:

intent of  "Defining Financial Yield
            Units of Measure"
  
    We are using a notation, where a number is
    followed by its unit of measure;
    for example:
    we  expect 0.7, to be equal to   70 percent 
    we  expect 0.01, to be equal to   100 bips 
   
    We also use some alternative names for
    the same concepts.

    we  expect 2 bips, to be a synonymous
      of 2 bps 
   
    These measures can also be used
    interchangeably with one another.
 
    consider that:
    we should  expect 2 percent,
       to be equivalent to   200 bps 

...

Now let's try something slightly tougher..
let's play with present value

intent of  "Specifying Money Currency"  
    First of all, we must be able to define Money
    we  expect 3, to be the "Money(3).amount"
    we  expect "EUR", to be "Money(3).currency"
    by default but we  expect "USD", to be
    the "Money(3,"USD").currency" 

...

intent of  "Using Cashflows"  
    Let's now deal with cashflows

    with  Conventional discount =
          continuously compounded yield

    we  expect about 271.67748, as the
        present value of 
                  Cashflow  [
                        100 at 1 year ,
                        100 at 2 years ,
                        100 at 3 years 
                  ]    ,
        with interest rate at 5 percent  

I don't know about you, but I can read this stuff out aloud with the expectation of being understood by domain experts.

A short index to the material of this and other blogs, in reading order.

Introduction

• Playing Between Sand and Water

Liquid Development

• Liquid Developers

Writing Styles

• Functional Style
• A Composition for Two Audiences
  

Shaping

• Shaping Code After Language I
• Shaping Code After Language II

Shaping VBA and Excel into an Agile Platform

• Excelagility - Part I
• Excelagility - Part II
  

The Nature of the Material

• The Web as a Medium
• Liquid Materials
• Nooclay I
• Nooclay II

Inhabited Software

• The Social Evolution of Software
  

Intent

• Instant Intent
  
• Intent and Proze I
• Intent and Proze II
• Intents, Stories and Facts
• Storytelling
  
• Evolving Maru Batsu from Intents I
• Evolving Maru Batsu from Intents II
• Distilling Intent From Descriptions
  

The Art of Growing Software

• Let Code DRY on its own I
• Let Code DRY on its own II
  

The Way of Meta

• The Way of Meta - Part I
• The Way of Meta - Part II
• The Way of Meta - Part III

Shaping Workshop

• Example Projects
  

Abstraction in Practice

• Time vs Distance

Perception

• Striving for a Fresh Eye
  
• Refactoring Emphasis in Language
  

Mastery

• Four Stages of Mastery I
• Four Stages of Mastery II
  

Projects

• RSpec DSL

Explorations

• Haskell
• Io and Boo
  

People

• Brian Marick
• Victor Cosby
• Stacy Curl
  

As a child, as many children do, I liked to play with sand on the beach, close to the sea.  I used to sit just a bit further away from the farthest point the longer waves would hit, and there I started to build.  I knew that, as the time passed, the odd wave would have hit my structure and that, with the passage of time, the whole structure would eventually dissolve, but I also knew that I would still have the time to see buildings, cities and civilisations evolve there, between sand and water, on the border of the sea.

I started out with big solid chunky lumps of sand, building up a solid irregular wall against the water.  In my mind those walls and those lumps of sand represented a primitive culture, just getting the jist of culture and civilisation, trying to ward off the entropic currents of the sea.

Then, on top and around those structures I started building more elaborate decorations.  The irregular buildings started getting a squared shape, and small walls started running between buildings.  This represented a young culture just discovering the value of functional shapes.

Inevitably a wave hit at one point, dissolving one wall and destroying part of my culture.  This destruction created vacuums where a new different culture or subculture could emerge and be shaped.  Depending on how it evolved and the waning and waxing of the waves, it could take over or disappear.

Baroque cultures appeared and took over the world, forgetting about the waves, building high pinnacles and towers into the sky, drawing beautiful arcs and digging tunnels close, too close to the sea… until they got destroyed almost at once by their own hubris.

More conservative cultures used most of their energies to built tall and thick walls to keep the water away.  The lack of water however didn't allow building much beyond those walls and, after a while walls and buildings just got dry and started crumbling.

Smart cultures were built around the very concept of wave.  They were shaped in such a way that the waves got channelled along canals into artificial lakes, turning the destructive force of water into a useful tool for further construction. 

However even smart cultures would become complacent and stopped building, stopped caring about new waves, sometimes coming from a different direction, sometimes of different size, and they also got eroded one bit at a time and disappeared.. leaving behind beautiful but unusable tunnels and artefacts for the newcomers.

The waves were, in my mind, the forces of time and change, the barbaric invasions and natural disasters, the forces of entropy continuously eating away at the culture.

More than twenty years later, while contemplating a kid playing on the beach of Antibes on the Cotes Azure, all these thoughts came back to my mind and I was surprised realising how close they were to development, the continuous evolution of structures, layers over layers of civilisation, always under siege by both the external and internal forces of entropy, always ready to fall to hubris and complacency.

And I realized with joy that I never stopped doing what I liked and what I wanted, I never really stopped playing with sand.