Strange Things are Afoot at the Circle-K

First of all, could we talk about an industry as a person? I am really starting to worry about Mr Software. Strange things are afoot at the Circle-K.

There are many smart people making software today. We got some geniuses don’t we? We make beaucoup bucks writing scribblies all day long. It really is the perfect crime. However, as flies swarm to shit, we have these MIT grads buzzing around financial institutions. We have geniuses with sun tans making guidance software for programs like “Drop it like its hot, Mr Gaddafi.”

As anyone who has filled out their taxes, or had to muddle through any government jungle, might have noticed that computers are now part of every little thing in your life. In Japan, they have toilets with more computing power than fresh air. Cars now have millions of lines of code, and when you consider how high bug densities can be per thousand lines of code, it might make you press the wrong peddle out of absolute defeat. Might as well end it now, before the software does!

Software is in a unique position in society. It can do stuff, and it’s everywhere. It touches the lives of everyone. However, we have these blokes deciding for themselves that being rich is cool, and it’s best to write software that skims the cream off the top of some morally reprehensible if not murderous dealings. Mr Software, your wife, she is not a hat.

You begin to wonder how these great minds can get by doing these bad and, at best, useless things. Are they evil? Were they conceived as vampires from Hitler’s blood? I really do not think so. Because we make software in context. We are a shape that is molded by the world. It is just that some people forget what shape they became.

Phil Zimbardo’s prison experiment (http://www.prisonexp.org/) demonstrated this artistic shaping of our selves. People really are natural born Thespians, and give them the chance, they will play the part you give them. You should visit some of the videos of the prison experiment, they will floor you. They are endlessly fascinating.

Even when we decide our shape is drooping and that we need to straighten ourselves, we have people like Stanly Miligram (http://en.wikipedia.org/wiki/Milgram_experiment) telling you that when the going gets tough, you will chicken out. But this is ok, because most do. I think the smart people call it ‘Rationalisation’.

So in the words of mighty Lenin, what is to be done? The world has so many problems to solve, how can we get people solving them. I mean, sure the MIT genius working for a financial company is solving problems. I guess, Mr Software, constipation is a problem too.

App Stores and the Soviet Union

App stores have turned the software business from an actual free market to Soviet style distribution centres.

There is a strange language that is used by these private corporations into thinking these stores they provide are some kind of free market. They call them markets when in reality they are more like your grocery store instead of a street market.

If you think shopping at a grocery store means you are shopping at a market, then I think we need to stop drinking the cool-aide. A grocery store picks which suppliers to buy from. The suppliers pick which items they sell. The government picks which food-stuff is produced cheaply through subsidies. And then finally YOU get to pick what to eat.

The problem with this is, the Soviet Union worked that way. You went to a government store and you get to pick what you want to buy. Then the information of what you bought went back to the central planning agency where they cranked this information in large computer systems to figure out if they are making what the people want. This sounds exactly like what Wal-Mart does. The only difference is you get to choose which “Wal-Mart” type store you go to which sells exactly the same stuff.

In fact any economy would have these kinds of distribution centres. Don’t let them fool you into thinking there is some free market system underlying it.

These App stores are distribution centres. Which item is bought, and who bought it is tracked very well. Then Apple, or Google, or whoever promote the most popular apps to the top, influencing what you buy in very obvious ways.

Now that the Mac App Store is here, and Google’s App Store, and Microsoft following suit. You have to stop and think when they use words like “market” and really evaluate what that means.

Do we want these private companies deciding what software you can buy? They already do on the computers you have in you pocket called cell phones. And the trend is obvious, they are moving into your living room too.

Look Like a Capitalist, Live Like a Communist.

“Look like a capitalist, live like a communist, fight like a fascist, and fuck like an anarchist.” is a great quote from a great short story from Miracle Jones.

We have a strange and kooky world here in software development land called Agile Development. Where we look like capitalists and live like communists. If you know anything about communism, and someone put a bit of Agile Development up to your nose on a hot plate while you were blindfolded, you would jump in delight. “Communism, my favorite working dish!” you will say. But when you open your eyes, you will see you really have a communist sausage wrapped in a capitalist bun. What is that all about?

How is it that the corporate world is now pushing us software people to work together in teams and decide for ourselves how to work. Its straight out of Trotsky’s pants. Its “Workers Control of Production”.

Well, kind of…

Sure, we get to deliberate among ourselves about the processes and tools. About who does what. About the code reviews and TDD. About how much foosball ball we can tolerate.

What we don’t get is the share of the profits. And we really, and I do mean really, don’t own anything.

Now of course you might be working for a startup and have stock options or be part owners. That’s great! Many great ideas and products were developed under communist conditions. Take for example Apple and their working out of the garage. Or Microsoft starting out of their garage too. These people worked like communists. They worked together as peers deciding what do to together. Eventually Apple and Microsoft got big and suddenly, the people who work there don’t own much at all.

So why are all these big companies pushing for Agile Development. I think its because they realize people work better when they decide for themselves how they should work. That is right, people work better in a communist way.

However, these companies hide this fact very well. They have terms like “Product Owner” so you know you don’t really own your work. They have gestapo type people called “Scrum Masters” who make sure you keep your nose down and create a buffer between the corporate world and your Eden of Agile.

This is either some fantastically brilliant capitalist plot to get people to work their best so they make the most money. Or this is some fiendishly lovely plan of some spy like communists getting in on the show floor through the back.

Discuss…

Theory of Relative Dependency and TDD

Refactoring is great right? I mean, it is something we do all the time. If you do TDD, you do it all the time, its part of the process. But is it such a good thing? What do we tend to do when we refactor and why in the world does code have “smells”?

I always felt that the idea refactoring is good was a sham. Our second bedroom for the longest time was my wife’s craft room. There was a big desk, and several other pieces of furniture. We installed shelves in various places, and generally the room had a certain amount of clutter to it. If I continued my dream of learning wood working, I might have wired the room for some wood working equipment.

Then my wife got pregnant, so we “refactored” the room. We had always expected to have a child, but never knew when that would happen. It became a real pain to change the room into a baby room. Clutter had to be removed, walls pained, furniture given away. Had we given the room more thought about its purpose and layout, the work to change it to a baby room would have been significantly less.

In other words, refactoring is the result of poor planning and poor design. How can poor planning be a good thing? I suppose in the modern world of “Agile” development using XP practices, down is up, and up is down.

After reading a paper by Koru and Emam about the “Theory of Relative Dependency” (IEEE Issue 99), got me thinking about refactoring and TDD in particular.

The paper was a study on many open source projects showing that for large scale systems, higher coupling was concentrated on smaller modules, and that smaller modules had a proportionally higher concentration of defects than larger ones. They also showed that refactoring exacerbates this coupling.

The TDD culture celebrates refactoring. The Test-Code-Refactor cycle is supposed to be lighting short. Empirical studies such as those by Janzen and Saledian do show that TDD does result is smaller modules with higher parameter “fan in”. Do you see where I am going? If you do, keep reading!

Koru and Eman recommended that agile software shops concentrate their QA and testing on these smaller components instead of the large ones because, duh, they have higher defect rates.

What is interesting with TDD is you get into this catch-22. You produce smaller modules and constantly refactor. These smaller modules will typically have higher coupling and defect rates according to Koru and Eman. Therefore you should focus your testing on these smaller modules!

You NEED the tests to sustain such rapid refactoring that TDD embraces. If you took some software designed by TDD and removed the tests, you would be left with many small modules that are highly dependent. This  is a difficult design to reason about and change. The software design of these systems will eventually collapse under their own weight.

The unit tests themselves are highly coupled pieces of code! And with a typical one-to-one ration between unit tests and code, this means half your code is tightly coupled code! Despite what others tell you, unit tests ARE code and they need to be maintained.

This frightens me. This means that any real meaningful refactoring , the cross functional, cross object kind would be almost impossible in a large project employing this approach.

I am trying to keep TDD out of my workplace for this reason. I feel that TDD is used by people as a crutch, as a replacement for thinking. Its friendly to that because you know what you are doing at any moment. “I am writing test now, ok it failed, not I write code, ok it passed, now I refactor, ok now I write more tests”.

Not thinking is built into the process.

For people who need this kind of structure, by all means do TDD. But for those adventurous enough to swim in the mental ether, to think, to feel, to care, avoid it.

A lot of this is speculative on my part and a lot of it is from my own experience with TDD. There are not many studies on TDD and many of those like Janzen and Saledian “Does Test-Driven Development Really Improve Software Design?” have mixed results. James Coplien wrote a great article called “Religion’s Newfound Restraint on Progress” that talks more about TDD and the dogmatic approach people take with it. I highly recommend it.

Using C++ STL Algorithms without Lambdas is an Advantage

A common complaint about C++ is that there are no lambdas (this will be fixed in C++0x), and most claim this makes STL algorithms like for_each, transform, accumulate almost useless because you have to create functions or functors somewhere else.

Not having lambdas is a GOOD thing. Although it can be annoying to create all these little functions/functors, there is an important advantage in doing so.

It forces you to think more modularly and you end up divide-and-conquering a problem.

It helps you think about your program on a higher level and also helps promote usability. Lambdas by definition are not reusable.

Here is an example

BoundingBox parseBounds(const string& bounds)
{
    //code that does parsing
    .....
    return result;
} 

vector<BoundingBox> createBounds(const vector<string>& bounds)
{
    vector<BoundingBox> result;

    transform(bounds.begin(), bounds.end(),
        back_inserter(result), parseBounds);

    return result;
}

If you use the STL algorithms you will be forced to write parseBounds. It is obvious that a function which converts a string to a BoundingBox will be useful in the future. If you wrote the loop yourself or had lambdas, you might have been tempted to write the conversion directly in createBounds.

When I write C++ I rarely write my own loops and I think it is a good discipline to pick up because it will force you to break your code up. It becomes almost impossible to write monolithic functions and also encourages code reuse. Best of all, your algorithms start looking simpler.

Prostitution

Is there something wrong with our field when prostitution is more enjoyable than programming?

Or is prostitution that enjoyable?

This is what it is

Basic blurbs about my ideas on software and the greater world.