Test-driven discipline

I’m going to hold an internal one-hour TDD training course tomorrow at work, and I plan to use this article as a reference. The majority of my co-workers has already taken part in a pretty great software craftsmanship training held by Kevlin Henney, so they should all be absolutely convinced of TDD. But understanding the value of TDD is not the issue here, making it work is. Therefore, I won’t tell people how great TDD is and why they should do it a lot more. Instead, I’ll give advice on how to stick with a test-driven approach without dropping efficiency, and possibly even more important, without losing your mind. I’ll do this by outlining the issues I had when I dipped my toes in the TDD pond. And because I failed the first time, I’ll also describe what happened the second time and how I managed to stick with TDD for the last decade after all.

Choosing a good code example

This is just a small part in a larger day-long TDD training that I’ve held several times and that covers the entirety of TDD’s greatness. Doing that has taught me that good quality code examples are key to a successful training, and it’s best to chose one that is simple. It also helps to chose from a domain that everyone can relate to, but that is not too well known, so it won’t spoil the surprises. I’ve therefore decided to go with something that tackles the issues of unit-testing and file system interactions. But it’s not a complete feature, since this would be too much to handle in just one hour. Instead, it focuses on handling files and directories in a way that allows test code to steer clear of the actual hardware during execution. It does not focus on actually reading and writing files, keeping the example on the shallow end of the complexity pool. The full source code can be found on GitHub, and I’ll add a link at the end of this article.

Test-First is not TDD

When I adopted a test-driven development process a decade ago, it wasn’t that easy, and I needed two attempts. My first one didn’t last more than a week and ended in complete failure across the board. I figured that all it takes is to write the tests before doing the implementation, but that didn’t work at all. Writing tests for an implementation that doesn’t yet exist is tough, especially when you’re still trying to write your tests for a specific implementation. I basically spent a lot of time doing the implementation in my head, and then I wrote the tests for that specific implementation. Eventually, I’d end up with the same result as I would without test-first, but it’d take much longer. The explanation for that is simple enough. Writing code in your head doesn’t give you compiler warnings, and you’re bound to change your design and adapt your tests in the process. Naturally, when trying out something new doesn’t work out as expected, people tend to stick to what they know. And being disappointed, I reverted to test-later.

Not too long after that, I gave it another try. As it turned out, my first ride on the test-driven train wasn’t that much of a failure after all. It gave me that essential insight, that just writing the tests first doesn’t impact your code at all, at least not the way I’d want it to. If you want your code to look different, you need to look at the problem that it tries to solve with a different angle. But that can’t work if you have a specific implementation in mind from the get go, and you’ll always try to make everything match the code in your head. It’s just the way our problem-solving focused brains work, so you need to give it something else to chew on while writing tests. For me, that something else is a very detailed specification that I can use to create an image of what I’m trying to achieve. Doing it this way shifts my point of view from that of a developer to that of a consumer, where I’m only concerned with how the feature in question is supposed to behave and look.

All that being said, TDD isn’t so much about writing your tests first. It’s about staying focused on the problem at hand and about always having your eyes on the ball. And the ball, for that matter, is the way a consumer will interact with your creation. It’s your public API that enables outsiders to leverage what-ever is implemented on the inside, but without any knowledge about the actual implementation. This should be the main focus. And, in contrast to the widespread opinions on TDD, everything begins with the initial requirement.

Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. But don’t forget about the consumer of your code, who might be a nuclear armed maniac.
John Woods (extended version)

Design phase

All developers know the problem of minimal requirements and rather unspecific specifications that leave very much room for interpretation. For the purpose of this article, I’ll start with a specification that most developers can relate to in terms of quality.

A file path shall be mockable to help with unit testing.
Requirement

As a [developer] I want to [mock file system related code] so that [it can be tested with unit tests].
User story

This is our starting point, and there is a lot to be done here before TDD has even the slightest chance to succeed. The most important thing right now is to get an understanding of what is asked. In this case, people want to mock file paths and file system related objects for unit testing. Obviously, accessing a file on the file system involves access to an outside resource, which a unit test can’t do by any definition. To connect both worlds, file system related code must abstract the calling code from the real thing, in this case the storage medium. But it’s not clear, from the given information, what kind of file system related objects need to be mocked. A file path is just a string in dotnet, and it doesn’t quite care about file systems. So if the code is working with file paths, then it’s not the path that needs mocking, but the mechanism that uses the path to access a resource that we don’t want to access during testing. Essentially, we need to mock the file system itself, or it’s object representation at runtime. But in dotnet, a lot of file system interaction is done using either static helper methods or instances of files and directories using BCL types like FileInfo and DirectoryInfo. There is nothing to mock here, and we’re doomed to keep using integration tests forever.

Well, not quite. TDD tells us to stay focused and to keep your eyes on the ball, so let’s define that ball. We can’t do anything about static methods, so these are out of the equation. The types FileInfo and DirectoryInfo, on the other hand, were introduced as a convenient alternative for those static methods. And since these are just runtime representations of real file system entities on a storage device, they can be regarded as glorified strings, containing a path that is nothing but an address in another medium. Unfortunately, our pals at Microsoft forgot to ship interfaces with them, which means that they can’t be used either, but we can define something similar and use that instead. With all that out of the way, our initial specification can be expanded. We now know that we’d rather use files and directories as objects, and we know that we need interfaces for them. And because we have the BCL types, we can also take an educated guess about the functionality that should be supported.

There shall be a dedicated contract that defines a file type.
There shall be a dedicated type that represents file instances at runtime.
File manipulation shall include: get name, get path, get if exists, get containing directory, create, delete, copy, move, read, write.
There shall be a dedicated contract that defines a directory type.
There shall be a dedicated type that represents directory instances at runtime.
Directory manipulation shall include: get name, get path, get if exists, get parent directory, get root, get children, create, delete, copy, move, create child

That was pretty easy, and we’ve got files and folders covered. It doesn’t solve the problem at hand, yet, but we’ve got the ball, and it’s ready to roll. The types that our files and folders are represented with can be cast into actual contracts in the code. A few more things need ironing out, though, such as input sanitation and error handling. Feeding null values as parameters into methods where it isn’t beneficial should throw the correct exception type, including useful information on what was wrong. Reading and writing files is best done using streams, so that’s what we’re going for without reinventing the wheel. If something went wrong during an action, a boolean return value is a good choice to indicate success and failure. And because we already know that there should be an abstraction between our types and the actual file system, we’ll just throw in a file system contract for good measure. But the file system should not be depending on our file and folder types. Instead, the contract should use BCL types, so that the development of entity types is not bound to the development of the various file systems and vice versa. As a result, a file system will not create and return instances of file system entities, but paths. In our case, we’ll use the already mentioned glorified strings FileInfo and DirectoryInfo.

Input sanitation shall throw applicable exception types with useful information, when an action can not continue safely.
File contents shall be accessed using streams.
The success of an executed action shall be made available through a boolean return value when possible.
A file system entity shall contain a reference to an abstracted file system.
There shall be a dedicated contract that defines a file system type.
A file system shall use BCL types to interface with a consumer.
A file system shall not create file system entities.
A file system entity shall not create file system instances.

One more thing that wasn’t immediately obvious from our initial specification is immutability. Paths used to be represented as strings, which are immutable in dotnet and which can only be changed when being set explicitly. The same thing should be true for our new types to reduce confusion and stick with the expected behaviour. In terms of file system contract and abstraction, specification can be kept short and simple. Basic file system interaction is more than enough to cover our planned functionality. Abstraction is done using different implementations of the contract, one for access to the real file system and one that can be used in tests and that is encapsulated inside the runtime. This also opens up more options, such as remote file systems that use protocols like SCP or FTP.

A file system entity shall be immutable after construction.
Actions in an instance of the fake runtime-only file system shall not affect another instance of the same type.
There shall be a dedicated type that represents the real file system at runtime.
There shall be a dedicated type that represents a fake runtime-only file system.
File system manipulation shall include: get if exists, create, delete, get contents of node.

It pays to add comments to your contract while you’re writing it, because that defines the exact behaviour of each individual member, according to the created specification from before. This is an important step that should not be omitted. Documentation in your contract acts as a single point of truth for everyone involved. Test authors use your documentation to cover the entire functionality in their tests, while consumers of your code use it to determine its functionality and behaviour. Everything that happens after this point is highly dependent on the contract documentation, and I can’t stress this fact enough. If you don’t have a well documented and properly specified contract, then you’re bound to think of the implementation when writing the tests for it.

/// <summary>
/// Defines an entity in a file system.
/// </summary>
public interface IFileSystemEntity {

	/// <summary>
	/// Gets the file system that the entity belongs to.
	/// </summary>
	IFileSystem FileSystem { get; }

	/// <summary>
	/// Gets the name of the entity.
	/// </summary>
	String Name { get; }

	/// <summary>
	/// Gets the full path of the entity within the <see cref="FileSystem"/>.
	/// </summary>
	String Path { get; }

	/// <summary>
	/// Gets if the entity exists within the <see cref="FileSystem"/>.
	/// </summary>
	Boolean Exists { get; }

	/// <summary>
	/// Creates the entity if it does not yet exist.
	/// </summary>
	/// <returns>True if the entity was created or if it already existed.</returns>
	Boolean Create();

	/// <summary>
	/// Deletes the entity if it does exist.
	/// </summary>
	/// <returns>True if the entity was deleted or if it didn't exist.</returns>
	Boolean Delete();

}

/// <summary>
/// Defines a leaf type file system entity.
/// </summary>
public interface IFile : IFileSystemEntity {

	/// <summary>
	/// Gets the directory of the file.
	/// </summary>
	IDirectory Directory { get; }

	/// <summary>
	/// Gets a read-only stream of the file.
	/// </summary>
	/// <returns>A read-only stream.</returns>
	Stream OpenRead();

	/// <summary>
	/// Gets a read-write stream of the file.
	/// </summary>
	/// <returns>A read-write stream.</returns>
	Stream OpenWrite();

	/// <summary>
	/// Copies the file to the new location.
	/// </summary>
	/// <param name="target">The new location.</param>
	/// <returns>True if the file was copied or if the new location matched the old one. False if the file didn't exist or if the new name is already taken.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="target"/> is null.</exception>
	Boolean CopyTo(IFile target);

	/// <summary>
	/// Moves the file to the new location.
	/// </summary>
	/// <param name="target">The new location.</param>
	/// <returns>True if the file was moved or if the new location matched the old one. False if the file didn't exist or if the new name is already taken.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="target"/> is null.</exception>
	Boolean MoveTo(IFile target);

}

/// <summary>
/// Defines a node type file system entity.
/// </summary>
public interface IDirectory : IFileSystemEntity {

	/// <summary>
	/// Gets the parent directory.
	/// </summary>
	IDirectory Parent { get; }

	/// <summary>
	/// Gets the root directory.
	/// </summary>
	IDirectory Root { get; }

	/// <summary>
	/// Gets all contained sub directories.
	/// </summary>
	/// <returns>A collection of <see cref="IDirectory"/>.</returns>
	IEnumerable<IDirectory> EnumerateDirectories();

	/// <summary>
	/// Gets all contained files.
	/// </summary>
	/// <returns>A collection of <see cref="IFile"/>.</returns>
	IEnumerable<IFile> EnumerateFiles();

	/// <summary>
	/// Gets all contained entites.
	/// </summary>
	/// <returns>A collection of <see cref="IFileSystemEntity"/>.</returns>
	IEnumerable<IFileSystemEntity> EnumerateFileSystemEntities();

	/// <summary>
	/// Copies the directory and all contained entities to the new location.
	/// </summary>
	/// <param name="target">The new location.</param>
	/// <returns>True if the directory was copied or if the new location matched the old one. False if the directory didn't exist or if the new name is already taken.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="target"/> is null.</exception>
	Boolean CopyTo(IDirectory target);

	/// <summary>
	/// Moves the directory and all contained entities to the new location.
	/// </summary>
	/// <param name="target">The new location.</param>
	/// <returns>True if the directory was moved or if the new location matched the old one. False if the directory didn't exist or if the new name is already taken.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="target"/> is null.</exception>
	Boolean MoveTo(IDirectory target);

	/// <summary>
	/// Creates a new child directory.
	/// </summary>
	/// <param name="name">The name of the new directory.</param>
	/// <returns>The new child directory.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="name"/> is null.</exception>
	/// <exception cref="ArgumentException">Thrown if <paramref name="name"/> is empty or white-space.</exception>
	IDirectory CreateDirectory(String name);

	/// <summary>
	/// Creates a new file in the directory.
	/// </summary>
	/// <param name="name">The name of the file.</param>
	/// <returns>The new file.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="name"/> is null.</exception>
	/// <exception cref="ArgumentException">Thrown if <paramref name="name"/> is empty or white-space.</exception>
	IFile CreateFile(String name);

}

/// <summary>
/// Defines a file system.
/// </summary>
public interface IFileSystem {

	/// <summary>
	/// Creates a new file at the given <paramref name="file"/> path.
	/// </summary>
	/// <param name="file">The file to create.</param>
	/// <returns>True if the file was created or if it already existed.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="file"/> is null.</exception>
	Boolean Create(FileInfo file);

	/// <summary>
	/// Creates a new directory at the given <paramref name="directory"/> path.
	/// </summary>
	/// <param name="directory">The directory to create.</param>
	/// <returns>True if the directory was created or if it already existed.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="directory"/> is null.</exception>
	Boolean Create(DirectoryInfo directory);

	/// <summary>
	/// Deletes an existing file at the given <paramref name="file"/> path.
	/// </summary>
	/// <param name="file">The file to delete.</param>
	/// <returns>True if the file was deleted or if it didn't exist.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="file"/> is null.</exception>
	Boolean Delete(FileInfo file);

	/// <summary>
	/// Deletes an existing directory at the given <paramref name="directory"/> path.
	/// </summary>
	/// <param name="directory">The directory to delete.</param>
	/// <returns>True if the directory was deleted or if it didn't exist.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="directory"/> is null.</exception>
	Boolean Delete(DirectoryInfo directory);

	/// <summary>
	/// Checks if a file exists at the given <paramref name="file"/> path.
	/// </summary>
	/// <param name="file">The file to check.</param>
	/// <returns>True if the file exists.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="file"/> is null.</exception>
	Boolean Exists(FileInfo file);

	/// <summary>
	/// Checks if a directory exists at the given <paramref name="directory"/> path.
	/// </summary>
	/// <param name="directory">The directory to check.</param>
	/// <returns>True if the directory exists.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="directory"/> is null.</exception>
	Boolean Exists(DirectoryInfo directory);

	/// <summary>
	/// Gets all existing sub directories in <paramref name="directory"/>.
	/// </summary>
	/// <param name="directory">The directory where to look.</param>
	/// <returns>A collection of directories.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="directory"/> is null.</exception>
	IEnumerable<DirectoryInfo> EnumerateDirectories(DirectoryInfo directory);

	/// <summary>
	/// Gets all existing files in <paramref name="directory"/>.
	/// </summary>
	/// <param name="directory">The directory where to look.</param>
	/// <returns>A collection of files.</returns>
	/// <exception cref="ArgumentNullException">Thrown if <paramref name="directory"/> is null.</exception>
	IEnumerable<FileInfo> EnumerateFiles(DirectoryInfo directory);

}

Development phase

That wasn’t too bad now, the contract is done, it’s documented, and it’s simple. There is a well-defined target, and we’re leaving the design phase. From here on, I will only use individual members to showcase the continuing workflow. The next big step is writing the tests that match our contract. For compiler happiness, we should add some implementing classes as well, but they should only throw a NotImplementedException. When writing tests for a member, I start with the error path and I end with the happy path. The reasoning behind this is that once your unit works, why bother writing any more tests? But exceptional behaviour is just as important, and I prefer to cover that first, so it won’t slip past. I’ll use IFileSystem.Exists(FileInfo) as an example for this stage, where I’ll write the tests according to the given contract documentation.

The file system tests should be written in a way that works for all implementations of the contract IFileSystem. The easiest way of doing this is by adding a using alias for a specific implementation in each test code file. Testing the isolation of the various file system implementations is the only implementation specific test code that is required. And if we’re going to run all test methods in parallel – which we should always want for reasons of isolation and performance – then there should also be a way of creating unique paths for individual testing.

using FileSystem = TestDrivenDiscipline.FakeFileSystem;

...

[TestMethod]
void ContextIsIsolated() {

	IFileSystem fs1 = new FileSystem();
	IFileSystem fs2 = new FileSystem();
	FileInfo file = new FileInfo(Path.Combine(_tempDir.FullName, "MyFile.txt"));

	fs1.Create(file);

	Test.If.Value.IsTrue(fs1.Exists(file));
	Test.If.Value.IsFalse(fs2.Exists(file));

}

internal DirectoryInfo _tempDir = new DirectoryInfo(Path.Combine(Path.GetTempPath(), $"UTESTS_{Guid.NewGuid()}"));

using FileSystem = TestDrivenDiscipline.RealFileSystem;

...

[TestMethod]
void ContextIsNotIsolated() {

	IFileSystem fs1 = new FileSystem();
	IFileSystem fs2 = new FileSystem();
	FileInfo file = new FileInfo(Path.Combine(_tempDir.FullName, "MyFile.txt"));

	fs1.Create(file);

	Test.If.Value.IsTrue(fs1.Exists(file));
	Test.If.Value.IsTrue(fs2.Exists(file));

}

// setup in ctor and tear down through IDisposable
internal DirectoryInfo _tempDir = new DirectoryInfo(Path.Combine(Path.GetTempPath(), $"ITESTS_{Guid.NewGuid()}"));

The fake file system can be tested with unit tests only, whereas the real file system does require a good amount of integration tests. Basically, everything beyond the error path of input validation is bound to be an integration test, since it touches the real file system. Only one in four scenarios will be unit-testable here. Obviously, someone could be providing a null value as file and that needs to be handled. This error case can’t be handled in a good way, so throwing an exception is in order. The BCL has us covered and provides the ArgumentNullException, that can also hold the name of the faulty argument in its ParamName property. If the provided value for file is not null, then the only other options are that the file exists, the file doesn’t exist but its containing directory exist, and neither the file nor its directory exist.

using FileSystem = TestDrivenDiscipline.FakeFileSystem;

...

[TestMethod]
void ExistsFile_Throws() {

	IFileSystem sut = new FileSystem();

	Test.If.Action.ThrowsException(() => sut.Exists((FileInfo) null), out ArgumentNullException ex);

	Test.If.Value.IsEqual(ex.ParamName, "file");

}

[TestMethod]
[TestData(nameof(File_Data))]
void ExistsNotExistingFile(String subPath) {

	IFileSystem sut = new FileSystem();
	FileInfo file = new FileInfo(Path.Combine(_tempDir.FullName, subPath));
	Boolean result = default;

	Test.IfNot.Action.ThrowsException(() => result = sut.Exists(file), out Exception _);

	Test.If.Value.IsFalse(result);

}

[TestMethod]
void ExistsExistingFile() {

	IFileSystem sut = new FileSystem();
	FileInfo file = new FileInfo(Path.Combine(_tempDir.FullName, "MyFile.txt"));
	sut.Create(file);
	Boolean result = default;

	Test.IfNot.Action.ThrowsException(() => result = sut.Exists(file), out Exception _);

	Test.If.Value.IsTrue(result);

}

internal DirectoryInfo _tempDir = new DirectoryInfo(Path.Combine(Path.GetTempPath(), $"UTESTS_{Guid.NewGuid()}"));

IEnumerable<Object[]> File_Data() {
	yield return new Object[] { "MyFile.txt" };
	yield return new Object[] { @"NotExisting\MyFile.txt" };
}

This is done for all the defined members, and I won’t stop writing tests until the entirety of the contract and its documentation is covered. This ensures, that I’m not starting to implement anything until I have a working indicator that tells me when I’m done. I realize that this approach is a bit extreme, but it’s the only thing that works for me, and I know from many previous clients that it worked for them too. Writing test first isn’t hard, writing the right tests first is. And to write the right tests, you need to have a very good understanding of how it should look like from the outside. I’ve defined the look and feel of the product code in the beginning, which has enabled me to write specific tests for every behavioural detail.

private readonly Dictionary<String, Stream> _fileCache = new Dictionary<String, Stream>();


public Boolean Exists(FileInfo file) {
	if(file == null) { throw new ArgumentNullException(nameof(file)); }

	return _fileCache.Keys.Any(_ => _ == file.FullName);
}

The implementing code in this case is a mere four lines of executable code, while the tests are weighing in at about five times as much. The numbers in most projects that I have worked on in a test-driven fashion are more along the lines of eight to ten lines of test code for every line of product code, but it’s worth to pay the price. The return is a safety net with a high initial coverage as well as greater testability of the product code.

Some people will argue that what I’ve described in this article isn’t TDD at all and that I should go and read the book. I strongly disagree here, because Kent’s book merely describes a specific development process and calls that TDD. But most people don’t get up in the morning, longing for yet another complex process in their lives (at least not that I know of). We don’t do TDD for the sake of the process, but because we want to increase testability, observability and other favourable properties of code. Just as all roads lead to Rome, there isn’t just one possible development process to get there. TDD encompasses all possible development processes that have this specific goal, and reserving it for just one of them feels a bit selfish. Kent Beck’s book is without the slightest doubt very valuable, but it’s not the bible. If it works for you without a hitch, be thankful and spread the word. But there are so many developers for whom it doesn’t work like that, and they deserve another way into test-driven development. This is another way. It may not work for everyone, but it’s worked for those who had trouble with following the book, so it can’t be that wrong, can it?

As always, the full source to the article can be found on GitHub.

mikelimasierra.net

Choosing a good code example

Test-First is not TDD

Design phase

Development phase

Leave a Reply Cancel reply