What we've been doing

Scaladays 2010

I attended Scaladays 2010 at EPFL in Lausanne, Switzerland in April. at which Nigel Warren and I presented the work we had done on porting Fly’s Java client library to Scala. You can watch all the Scaladays videos on the Scaladays website and so I will not dwell on the talks, which were excellent. Instead, I want to talk about Scala and its community, led by Martin Ordersky.

It has become clear to me, as it did with Java 15 years ago, that Scala is an incredibly important language and almost certainly Java’s successor. There is no reason for anyone intending to build applications for the JVM to choose Java over Scala.

Any Java developer is capable of writing Scala in a basic sense, that is, write exactly what they would write in Java but in Scala’s syntax. They can even start by mixing Java and Scala if they feel uncomfortable about jumping all the way in. Developers can choose to use Scala on parts of a system or even just in tests. Scala’s tight integration with Java libraries and frameworks enables developers to make use of the Java ecosystem as they normally would which is incredibly valuable.

However, Scala has a great deal of headroom. It enables much more concise, simpler, reusable and elegant software to be written for those that have the will to learn functional programming and a little practical type theory. Importantly, Scala offers Java developers a way to move from idiomatic Java to idiomatic Scala in small steps, there is no need for a wholesale leap into the unknown. In short, Scala offers a migration path to a better way to build robust software.

A reason why Scala is important is that it offers mechanisms for controlling complexity. In The Structure and Interpretation of Computer Programs, Abelson and Sussman said:

In our study of program design, we have seen that expert programmers control the complexity of their designs with the same general techniques used by designers of all complex systems. They combine primitive elements to form compound objects, they abstract compound objects to form higher-level building blocks, and they preserve modularity by adopting appropriate large-scale views of system structure.

Scala facilitates this abstraction and combination that results in simpler, more robust software in a number of ways: functional programming techniques, object oriented techniques, a rich type system and mechanisms to build components. In Scalable Component Abstractions, Martin Odersky and Matthias Zenger describe abstractions for the construction of reusable components:

abstract type members, explicit selftypes, and modular mixin composition. Together, these abstractions enable us to transform an arbitrary assembly of static program parts with hard references between them into a system of reusable components.

Making full use of Scala does not come for free, there is a lot to learn. Fortunately, the Scala community is very friendly which is in large part due to Martin and his team’s efforts. The community is full of very smart people who are very willing to help novices. It is not unusual for questions asked on mailing lists to be answered by Martin or other leaders in the Scala community. This friendly environment is actively fostered so that beginners feel comfortable asking basic questions without fear of angry replies to RTFM.

There are those that think that Scala is a complex language and they are right to a certain extent, but you can be productive in Scala without using advanced features, just not as productive as you would be if you learnt about those features.

Most objections to new languages like Scala and Clojure can be reduced to, “I don’t understand this after 2 mins of skimming the website so its too academic”. Not very good really and not worthy of further argument.

Learning Scala has real benefits in understanding more about computing in general, and it lets you do it gently whilst being productive. Give it a try!

Writing Maintainable Acceptance Tests

Over the past six months or so, there has been a fair amount of negative commentary about automated acceptance / integration / system testing. The thrust of this commentary is that testing at this level tends to be brittle, slow and have a high maintenance overhead. None of this needs to be true, but producing a robust suite of tests requires an uncommon adherence to good practice.

Read the rest of this entry »

Large Systems

Many times over the years we have encountered an attitude towards code cleanliness that is summed up in the assertion that as systems grow in size, code quality will necessarily degrade. Often the argument is backed by references to ‘programming in the large’ as opposed to ‘programming in the small’. We have consistently found these assertions to be unfounded and this article is an attempt to explain why.

Read the rest of this entry »

Producing Systems that Do Not Rot

I was driven to write this article after reading Kirk Knoernschild’s blog about Rotting Design and felt I needed to say something.

Read the rest of this entry »

There Is Only One Codebase

Some Casual Miracles consultants have recently been helping teams in which the central, unrecognised problem was that there were multiple streams of development that operated on different timescales (i.e. release dates were uncoordinated) but which shared a common codebase.

Read the rest of this entry »

Software Quality – Pragmatics

This article continues my series on Software Quality. My thesis is that we judge quality using the same inbuilt sense of aesthetics as we would a work of art or an everyday object, albeit at quite a high level of abstraction.

The design industry has a lot to say about this kind of thing and no one more so than Massimo Vignelli, who summarised his core ideas in The Vignelli Canon. I am following each part of the Canon, this part being Pragmatics.

Whatever we do, if not understood, fails to communicate and is wasted effort. – Vignelli

No matter how cleverly we design something, if it is not understood then that design has failed. Software is complex and some decisions will need explanation, but if the design is still not understood then the effort in producing it is wasted.

Complicated, confused designs are very common in our industry. They are often the result of confusion in the minds of their developers, analysts and users. A confused mind will produce a confused product. These projects usually limp along before collapsing, mercifully, under their own weight, but not after many years of very expensive and wasted effort.

We should be careful not to confuse a complex solution with an accidentally complicated one. Some systems are unavoidably complex and what we expect to find there, as in all systems, is a set of core principles that underpin the whole edifice. Without clear, strong and forceful decisions that provide structure to systems, there will be no clarity and plenty of confusion. This is not to say that those decisions are carved in stone. Constantly changing environments and requirements are a fact of life in software and so those decisions will need to be revisited and reworked. However, it is much easier to move from one set of clear and strong design decisions to another, than it is to rework a confused system.

Having clear, strong and forceful decisions is the only pragmatic choice we can make. Allowing ambiguity and confusion is not pragmatic, it is simply sloppy and is the easy way out in the short term.

Vignelli sums it beautifully:

• We like Design to be forceful
• We do not like limpy Design
• We like Design to be intellectually elegant – that means elegance of the mind, not one of manners, elegance that is the opposite of vulgarity
• We like Design to be beyond fashionable modes and temporary fads
• We like Design to be as timeless as possible
• We despise the culture of obsolescence
• We feel the moral imperative of designing things that will last for a long time

This is pragmatism in any field, and no more so than in software.

TDD from a Control Theory Point of View

Before becoming a developer, I was an academic at University College London. We built the world’s first homodyne and heterodyne phase locked loop with semiconductor lasers for oscillators instead of electronic oscillators. Sounds easy, right? But the problem is that because lasers oscillate at high frequencies (it’s light after all), a delay of a few hundred picoseconds around the feedback loop is significant and results in the system becoming unstable.

Indeed, any system with feedback can become unstable if the delay around the loop is significant. There are things that one can do to prevent instability but the best thing to do is remove the delay if possible.

In software, there are many feedback loops. When you write code, often the first feedback you get is whether it compiles or not. Can you imagine the pain if that wasn’t the case? Waiting a few weeks to find out whether the code actually compiled would be ridiculous. Right?

Fortunately, most software developers have a go at making sure their software actually compiles. Some don’t, but thats a different article.

Testing is another feedback mechanism. So how is it sane to wait months before finding out that your code actually works? It isn’t, thats crazy talk. TDD helps in a number of ways: it helps you to think about what your code will do before you think about how it will do it; it provides a safety-net for all those refactorings you will need to do; and it tells you that the code you thought you wrote is what you actually wrote.

The instant feedback obtained by TDD is a stabilising mechanism to ensure that what is written is correct and that you haven’t broken anything else inadvertently because all that other code in the system is also tested.

But what is meant by stability in this context?

The main instability is the code – wait – test – fix – repeat cycle. During the wait and test phases, new code is being built on top of the broken code, and the fixes themselves often introduce new defects. This results in increasingly painful fix phases. What is worse is that there may be many overlapping loops like this which is utterly chaotic!

This chaotic process results in developers working in a panicked, crisis mode which leaves no time for refactoring, reflection about improving the codebase, or writing tests. It’s very demoralising as the deterioration in the codebase cannot be stopped.

The end result is a brittle, sprawling mess of a system that is now not in any state to be tested or repaired without heroic effort from the developers, and buy in from business stakeholders. And we know how that story ends…

Different Interpretations

As a retrospective facilitator I frequently see different parts of a team presenting different perceptions of the same event. In fact, this underlies a great many of the problems seen in teams.

Read the rest of this entry »

Demand Quality: You Get What You Ask For

“In the long run, men hit only what they aim at. Therefore, though they should fail immediately, they had better aim at something high.”  – Henry David Thoreau

Read the rest of this entry »

Fly for Scala 1.2.4

The latest version of Fly for Scala has been released and is available at http://www.flyobjectspace.com/.

Read the rest of this entry »

Site Designed By Top Left Design