Refactoring is great right? I mean, it is something we do all the time. If you do TDD, you do it all the time, its part of the process. But is it such a good thing? What do we tend to do when we refactor and why in the world does code have “smells”?
I always felt that the idea refactoring is good was a sham. Our second bedroom for the longest time was my wife’s craft room. There was a big desk, and several other pieces of furniture. We installed shelves in various places, and generally the room had a certain amount of clutter to it. If I continued my dream of learning wood working, I might have wired the room for some wood working equipment.
Then my wife got pregnant, so we “refactored” the room. We had always expected to have a child, but never knew when that would happen. It became a real pain to change the room into a baby room. Clutter had to be removed, walls pained, furniture given away. Had we given the room more thought about its purpose and layout, the work to change it to a baby room would have been significantly less.
In other words, refactoring is the result of poor planning and poor design. How can poor planning be a good thing? I suppose in the modern world of “Agile” development using XP practices, down is up, and up is down.
After reading a paper by Koru and Emam about the “Theory of Relative Dependency” (IEEE Issue 99), got me thinking about refactoring and TDD in particular.
The paper was a study on many open source projects showing that for large scale systems, higher coupling was concentrated on smaller modules, and that smaller modules had a proportionally higher concentration of defects than larger ones. They also showed that refactoring exacerbates this coupling.
The TDD culture celebrates refactoring. The Test-Code-Refactor cycle is supposed to be lighting short. Empirical studies such as those by Janzen and Saledian do show that TDD does result is smaller modules with higher parameter “fan in”. Do you see where I am going? If you do, keep reading!
Koru and Eman recommended that agile software shops concentrate their QA and testing on these smaller components instead of the large ones because, duh, they have higher defect rates.
What is interesting with TDD is you get into this catch-22. You produce smaller modules and constantly refactor. These smaller modules will typically have higher coupling and defect rates according to Koru and Eman. Therefore you should focus your testing on these smaller modules!
You NEED the tests to sustain such rapid refactoring that TDD embraces. If you took some software designed by TDD and removed the tests, you would be left with many small modules that are highly dependent. This is a difficult design to reason about and change. The software design of these systems will eventually collapse under their own weight.
The unit tests themselves are highly coupled pieces of code! And with a typical one-to-one ratio between unit tests and code, this means half your code is tightly coupled code! Despite what others tell you, unit tests ARE code and they need to be maintained.
This frightens me. This means that any real meaningful refactoring , the cross functional, cross object kind would be almost impossible in a large project employing this approach.
I am trying to keep TDD out of my workplace for this reason. I feel that TDD is used by people as a crutch, as a replacement for thinking. Its friendly to that because you know what you are doing at any moment. “I am writing test now, ok it failed, not I write code, ok it passed, now I refactor, ok now I write more tests”.
Not thinking is built into the process.
For people who need this kind of structure, by all means do TDD. But for those adventurous enough to swim in the mental ether, to think, to feel, to care, avoid it.
A lot of this is speculative on my part and a lot of it is from my own experience with TDD. There are not many studies on TDD and many of those like Janzen and Saledian “Does Test-Driven Development Really Improve Software Design?” have mixed results. James Coplien wrote a great article called “Religion’s Newfound Restraint on Progress” that talks more about TDD and the dogmatic approach people take with it. I highly recommend it.
Mempko 
Actually I think this is refactoring the way consulting firms use it. To them the goal is to increase the code size and complexity to a point where nobody will ever be able to work on the system without their help. The more indirection and coupling the better. That leads to “refactoring” that increases bloat, adds complexity, causes busy work, and creates the situation described by Eman and Koru.
If however, you refactor to always *remove* code, and actually try to measure this, then you avoid the problem. If I do something I think is improving things, and then I go look at the diff and it’s just adding a bunch of code and no real features, then I just revert it and don’t do it.
Try it next time. Before you commit, take a look at the diff and then justify it if it increases the code size.
I agree with you completely that if a specific refactor can reduce code, then it is a worthy effort. However the TDD culture seems to view refactoring as an end in itself. Supporters use “ease of refactoring” as an argument for using TDD. That is plain silly and I think you would agree. I would argue further that the kind of refactoring that reduces code is often cross functional and spans components, which TDD may actually make harder because of the sheer amount of test code you would have to change.
You admit a lot of this is speculative. I’d encourage you to keep exploring some of your assertions:
“If you took some software designed by TDD and removed the tests, you would be left with many small modules that are highly dependent.”
“This means that any real meaningful refactoring … would be almost impossible … employing TDD.”
“Not thinking is built in to the process.”
I’d also support you in removing comments like this from your post, it sounds like an ‘appeal to ridicule’ fallacy:
“For people who need this kind of structure, by all means do TDD. But for those adventurous enough to swim in the mental ether, to think, to feel, to care, avoid it.”
It’s not speculation as I have seen this personally. Though it IS opinion and it is speculation that TDD was the “root cause”. However, it is just as much a gut feeling that TDD imroves your code. Very few actual studies look at this and many of them reach conclusions that are not in line with TDD proponents.
While I do find that the test code is coupled to the code it tests the act of TDDing something reduces coupling from other parts of the system enormously. It has this impact because otherwise the tests are unnecessarily complex, and the process of continuously improving the test and the code reduces those external couplings to simple interfaces – without such the tests would be too complex.
To me TDD is the process of thinking about the design of a feature, a module and a class before I write it. I get to play with the various ways in which I might code it and see which seems to fit best, but doing so without ever really implementing it, just using it externally.
So far I’ve seen nothing but enormously positive impact on a teams progress and bug count after adopting TDD well. It of course doesn’t suit everything, I wouldn’t advise using it for safety critical work or high performance embedded work as I would hope they have an even better design and testing process. But it does work on enterprise software and websites especially well.
When I was first introduced to refactoring, it was provided as the alternative to the “start over and rewrite the code again” standard practice of the time. Refactoring was not repurposing of code — it was about improving the code without changing the observable behavior.
Thus the unit tests. Your tests pass before, then you refactor some code, and your tests pass after, and the code design is incrementally improved while the observable behavior is unchanged. If you’re changing observable behavior, you’re not refactoring.
(Your baby-room example is flawed. Had you been ‘refactoring’, you would have made a small change every time you walked into the room over the course of your wife’s pregnancy: call it eight months of moving one piece of clutter ever time you walked into the room — the christmas ornaments would have ended up in the attic, the winter clothes would have been boxed up and put in the garage, the leisure suit thrown away, the IBM XT in the office ‘gifted’ to the goodwill so that the sewing machine could be moved into the office… For real rooms, that’s a hard way to do it, but for code, it’s much easier.)
As for TDD, my thought has always been that it’s a good way to train junior and mediocre programmers to design their code so that it CAN be tested. I’ve worked with a lot of programmers who routinely made their design decisions in such a way so as to make the code very difficult to test in isolation. By making these programmers write tests *first*, they (hopefully) learn how to design their code in such a way so that it can be tested — eventually, one hopes, where they don’t have to actually write the test first.
I’ve worked with a lot of systems that were not designed to be tested at all, and not only did their design collapse under its own weight, the established codebase was so fragile that the developers tasked with maintaining it were afraid to touch much of anything, because nobody knew what interactions would cause the system to fail. Consequently, instead of *solving* problems, layer after layer of workarounds were added, making the codebase more fragile, and compounding the problem.
I think your attempt to keep TDD out of your workplace is misguided. Perhaps you don’t need to write the tests /first/, but until you can write code where writing the unit tests immediately afterward (without requiring sophisticated mocking frameworks), you should train yourself and your team by writing the tests *first*. Then, when you can design your software so that implementing the tests is a trivial task that can be left up to an intern, you can eschew the TDD approach.
After all, the problem is people not thinking, not TDD. If you have people on your team who aren’t thinking, you’re doomed regardless.
The one thing TDD does teach you is to design your code so that it can be isolated (i.e., loosely coupled) and tested. If you and your team can already do this, congratulations. If anyone on your team can’t, then your team ought to do a little TDD until everyone on your team CAN design in this way.
I agree with you completely Annoyed. TDD is really good for junior programmers. It does help them quickly use their designs. Tests are after all just tiny programs. And you are right that non thinking is the problem, not TDD. However TDD supporters often call people lazy and say you are not a professional if you are not using TDD. This is my point, that TDD is not a substitute for thinking and it seems you agree.
I’m confused. You agree with Annoyed, and he just said TDD teaches you to design your code to be _loosely_ coupled, but in your article you say, “If you took some software designed by TDD and removed the tests, you would be left with many small modules that are highly dependent.” Which is it?
Also, it sounds like in your post, though, that not thinking is inherent to TDD, but now … do you think one can practice TDD and be a thoughtful designer?
Have you ever watched Karate Kid? Wax on, wax off. TDD does help junior programmers create code that is loosely coupled. And it does help them think about their design. But its not because that’s inherent to TDD, but because they are junior programmers. They will usually be in school writing small programs, or they would work on a very small system. They will not likely see large “fan-in”, and they will not likely do major cross component refactoring because the are more likely to work on small systems. Same reason it helps them think about their code. Wax on, Wax off. However, just about any other disciplined process can help them think about their code in my opinion.
So your statement about looking at code made by TDD being highly coupled only applies to large systems? If so, I didn’t pick this up from your post where it sounds like you believed TDD would result in tight coupling regardless of the size of the system (fwiw).
Another point of confusion for me: you say both, “refactoring is the result of poor planning and poor design” and “This frightens me. This means that any real meaningful refactoring, the cross functional, cross object kind would be almost impossible in a large project employing this approach.”
Do you think refactoring is a valid practice itself or should be done away with? If the latter, is there such a thing as a meaningful refactoring in your mind?
It’s not convincing to say “left with many small modules that are highly dependent” is a result of TDD. I think more and more small modules is the consequence of pursuing flexibility – you’ll get many small modules even you don’t follow TDD.
I think in some parts you’re being so spiteful about TDD. when you try to write unit tests, don’t you think you need to think clearly about what you need?!!! in TDD you look at things from different point of view. you think about what you need and about how you can do it in the best way but maybe(surely) at that moment you can’t think of the best way, so you’ll have the code and functionality under test and you’ll refactor it through the path to the better way incrementally over and over again until you’ll reach at the best solution for the given problem. but if you don’t have test how can you be sure that you’re not breaking anything while you change things for making them better. cause we all know that 1st solution is not the best even 2nd and 3rd, so you have to make things better all the times and I think it’ll be so hard without tests and if you do TDD you’ll surely have better tests and isolated parts which will be easier to handle and improve.
This works perfect for small and large projects.
I have nothing against unit tests. I do them. The thing is there is a lot of work you have to do before you start writing code. You have to research the domain, talk to experts, use familiar concepts. Then you can make a lightweight architecture and design. You can then use unit tests if you want to flesh out the details of what those components look like.
unit tests are after all are little focused programs.
However there are better ways in making sure you don’t break anything with changes, and I think design by contract will better find bugs and the compiler will find the obvious ones.
The definition of refactoring according to Fowler: “Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior…” You seem to be missing the part about not changing behavior. Changing the purpose of a room is not a refactoring. What you did is a big rewrite. Those often fail. You can’t expect it to be easy to totally re-purpose something.
I would cringe if asked to turn my living room into a kitchen. On the other hand, if you asked me to redo my living room as a more maintainable living room then I could write a list of assertions defining what a good living room is and slowly refactor my living room to be more awesome while still meeting all of the criteria outlined in my tests. On one day I might rearrange the seating to allow for faster navigation around the room. On another day I might replace the carpet with something that needs to be cleaned less often. It’s still the same room with the same purpose but it achieves that purpose better than it did before.
Also, nothing about TDD requires that you build highly coupled systems with too many small pieces. Bad practitioners can muck up code using any paradigm. Good program designers will use their tests as a net to support the generalization of their code over time so that it is only as coupled as it needs to be. For example, Dependency Inversion is a concept many developers embrace to write loosely coupled systems. TDD works just fine in a system based on dependency inversion.
Nothing about TDD requires you to be mindless. If you are mindless while doing the TDD cycle then your code will probably suck, but it would have sucked no matter what techniques you were using while developing. An implicit assumption behind any process you use while developing should be mindfulness. I would have a hard time considering anyone a professional if they are not mindful while writing code. There are an easily enumerable number of rules for playing chess. That doesn’t make chess a mindless game.
It pains me to see that you are arguing that TDD will somehow make a good developer write worse code. How can testing your beliefs result in a less useful and maintainable system? I develop professionally on some really large systems. Some of these systems have great tests and some do not. The ones with tests are trivial to update. The ones without tests eat up a disproportionate amount of our time and break way more often. Some of them had tests but were not test driven. Those systems tend to decay over time. The systems that are test driven can never decay because there is never any period of time where the test suite doesn’t pass after a round of changes is committed.
This is one of the best comments I have received. It is well thought out and well written. However I think you are taking my example about the room too literally. All metaphors break down in detail.
You are absolutely right about what refactoring is. It is restructuring code without changing the behaviour. There is no argument here.
I can of course continue with my room metaphor and claim that indeed I am not changing the behaviour of my house. The various crafts are distributed to other rooms that will serve those functions. Now suddenly the metaphor still works because the existing behaviour of my house is still the same.
So this brings up a fairly interesting point. We mostly refactor when we need to add new functionality to the existing system. And you are absolutely right that you must refactor in order to prevent decay.
My whole point is that if you do a little thinking ahead of time, you can decide what is easy to change, and what is difficult early on, instead of everything being difficult to change.
With TDD you end up with twice as much code to change when you refactor. In practice tests are code too and cannot be looked at differently.
What I would recommend over TDD would be Design by Contract. This still gives you the confidence that the system behaviour doesn’t change when you refactor, but also has the advantage of also detecting integration and system wide behaviour bugs.
Design by Contract also gives you the same advantages TDD does, such as thinking about the design of your interfaces. However it wins out because it not only documents code in context, it also works by checking that internal program state remains predictable, and the code fails fast if something is wrong. With TDD, these assertions are far removed from the code and often only test a subset of the interface’s domain.
Design by Contract is great, I wish I had more opportunities to use it. I suspect that I would still TDD in a language with solid DbC support. I like having an easily runnable suite of tests that I can invoke at any time to confirm my beliefs about the system. If the system is not built in a test driven fashion then it will (in my experience) most likely have less than ideal automated code path execution coverage. DbC won’t help you much if you don’t have a simple and fast way to invoke all of your code. Compile time DbC checks will catch many issues but it is not possible to validate all conditions in this way. My experience with DbC is relatively limited, apologies if I have misspoken.
In practice I find that having a large suite of tests isn’t much of a pain when my design has to be reworked. Automated tools can achieve many refactorings with a few key presses. I use vim and have plugins that allow me to execute the most common refactorings with ease. When requirements change, or I have an idea on how to improve the design, I make the necessary transformations and have tests to confirm that I did not unintentionally break anything. Test code must be written with the same level of quality as production code. If that happens then it should not be brittle or painful to upkeep.
Test driving works for me. I end up with better code. I feel more confident along the way. I don’t waste time writing our flawed algorithms. I find it easier to pick up the code six months later and feel comfortable making changes.
Stepping back to look at the big picture, it seems as though we agree on a lot and disagree on a lot. Seems as though we will have to agree to disagree. Best of luck.
Actually I think if you look at the big picture we agree on the most important points. We agree that if you work with mindfulness, with care at every step, you will get the best results.
Whether or not TDD is the best approach to help people work mindfully is not set in stone. There is simply not enough evidence for me to say that TDD should be the approach used. And I think it would be intellectually dishonest to claim the gains that most proponents express TDD gives you. I can’t say Design by Contract gets you those gains either, and I won’t.
We also agree that crappy programmers will do crappy work in any paradigm.
If you look really closely, Design by Contract and TDD do very similar things. They just have very different real world consequences.
The assertions in Design By Contract are written right where the code is written. In TDD, they are written in an example program called a unit test, and are checked at the point of call. Design By Contract assertions have access to private members for free while TDD you must refactor a class to pull that private information into a checkable way (this might be considered a TDD advantage). Design By Contract assertions are run every time the program is run, in real world input while TDD tests are run periodically on fabricated input. Both Design by Contract assertions and TDD tests fail fast.
Once again, there is not enough research in Design by Contract to make any definitive claims that Design by Contract works. There is some like this microsoft study (http://research.microsoft.com/pubs/70290/tr-2006-54.pdf). However, there is also not enough research in TDD to show that it accomplishes any of its claims.
An argument can also be made into using both techniques as complimentary. Use TDD to help design the interface, and use Design by Contract to help understand the state and logic of the program, and make sure the system works in the real world.
I came across this site while doing some research on the Internet tonight. It is great to hear such valuable opinions. While I am not an expert on TDD, I would like to comment on two points about our own work for clarification purposes:
First, smaller modules are not more defect prone or more dependent; they are less defect prone and they have less dependencies. However, proportionally (per line of code), smaller modules have more defects and more dependencies compared to larger modules.
Secondly, I would like to stress that our research results cannot inform software design process. Based on the empirical evidence, the theory of relative defect proneness and the theory of relative dependency simply state what it is and they make predictions about certain relationships in large-scale and real-life software products. If the theories hold, knowing these relationships can be a powerful weapon when it comes to applying focused and prioritized efforts for quality improvement (e.g., intense inspection and testing efforts done in a week before shipping the next release).
While refactoring was found to be a potential explanation behind the observed relationships, it is important to note that refactoring techniques are very useful to improve the overall design. Generally speaking, refactoring is needed and it should not be avoided. This point was also stressed in the above mentioned article. For example, refactoring can increase the dependency concentration in smaller modules but, at the same time, along with its many other benefits, it decreases the overall size of a software product by improving the within—product–code-reuse.
Time will show whether the observations of our distinguished practitioner colleagues such as yourselves and those of our fellow researchers will verify these theories. So far, our research results have been consistent but, even though many products were analyzed, we need more replicated or closely-replicated empirical studies on the same questions preferably conducted at many different research sites.
Of course, software practitioners do not (and, they should not) blindly apply any advice before giving a careful thought about their particular conditions. Software development is a complex setting and there will be many other concerns, such as the business importance of software modules, variation in developer skills, etc. etc.
Best wishes,
Gunes Koru
Gunes Koru,
Thank you very much for clarify your work. It helped me further understand your paper and I completely agree with your judgements.
I hope that I have not implied your research had anything to do with TDD. Your paper gave me an insight into what I saw with software written strictly using TDD. The relationships between modules that your paper addresses helped be understand the behaviour I saw in software written using TDD.
I am also not arguing against any refactoring that decreases code size (in other words, decreases absolute defects, see my reply to Zed Shaw). I actually implied that TDD exactly hinders this kind of refactoring.
I had a great insight reading your paper which confirmed what I saw with code written by the TDD process. This code is refactored, but not for the sake of re-usability and overall code decrease, but because it has to be, because it was not designed correctly from the beginning. TDD process often forces code breakup without regard to design. Though the end result is often argued by TDD proponents as good design.
And thank you emphasising that practitioners should not blindly apply any advice before giving careful thought. Unfortunately TDD is a sparsely researched and unproven process, but practitioners follow it anyway because an “expert” says that professionals must do TDD. I feel that you would agree with this sentiment.
Software is complex, and research such as yours which look at the complex relationships within software help further the profession, and help deepen our understand of this complexity.
Thanks!