Returning to the table, Dan greeted Kate and asked her “How do you do that?”

“Do what,” Kate said.

“Always arrive at meetings without anyone seeing you coming and going. And with your drink, even!”

“Well, I don’t always do that. I guess sometimes people just aren’t very observant, though with my red hair, robust figure, and sparkling attitude, you’d think I’d be hard to miss. Maybe I’m part ninja.”

“Robust, is that what you call it? Anyway you’re not my usual image of a ninja,” Dan said.

Kate laughed. “Some kind of bust, anyway. Thing is, I’m in disguise. What good is it to be a ninja if you look like a ninja?”

The chat continued for a while. Then Dan got down to business. “Kate, you’re doing a wonderful job at Oak River. Things are turned around there, and I’ve been able to put most of my attention on other matters. One of those is that I’m looking at acquiring another company in our line of work. They have a good customer base, but haven’t released much at all in over a year. I’m thinking a bit of your magic might help them out.”

Kate said, “It’s not magic, but it might help. I think what we’re doing will help any organization that does what we do. And, by the way, it won’t help anyone who doesn’t do it, the same way that a diet won’t help people lose weight if they don’t live by it.”

“That makes sense,” Dan said.

“You’d think so. But I read every day about teams who say these things won’t work, but have never really tried them. I wonder how they know.”

“What are you doing, anyway, Kate? Is it Scrum? What is Scrum, anyway?”

Kate took a sip of her drink and thought a moment. “We’re doing something like Scrum, which I’ll describe. Let’s start with what Scrum is. I’ll blather on a bit but feel free to stop me with questions if you need to.

“Scrum is a framework, a way for people to work, a style, you might say, for building products. I think of it as a beginning, a good starting point for building something. If a team will start with Scrum, and pay attention, they’ll see the things that are in their way and be in a position to improve. Naturally, some of the things that are in a team’s way are outside their direct control, and in those cases, management and the organization have to be responsible for removing what Scrum calls the ‘impediments’ that are in the way.

“There are just three ‘roles’ in Scrum: the Product Owner, who determines what the team will build in the next short interval or ‘Sprint’; the Team, who build it; and the Scrum Master, who facilitates the process and makes sure that the process, and the people, continually improve. There are four meetings, Sprint Planning, Sprint Demo, Sprint Retrospective, and Daily Scrum. And there are just three artifacts, the Product Backlog, the Sprint Backlog, and the Increment.”

Dan said, “So the Scrum Master is the manager, and the Product Owner is an analyst?”

“No,” Kate said, “Not at all.

“In a Scrum project, the Product Owner is responsible for delivering the best possible product within time and budget. The team builds what she says to build, and her job is to get the best possible results. She can do this because the team shows her real running product every Sprint. That is, every couple of weeks, thirty days at most. She’s in a good position to see how fast things are going, and to make the big decisions as well as the smaller ones.”

“So the Product Owner is the manager, then,” Dan said.

“No, Kate said, “Not at all. In fact, a Scrum team need not have a manager at all. In Scrum, the team is ‘self-organizing’. To as large a degree as possible, they manage themselves. Naturally, in large companies, or companies with a lot of ‘managers’ running around, it is common for a Scrum team to have a manager. Frankly, this is often harmful and it is a common cause of Scrum not working very well.”

“But who decides things,” said Dan. “Who decides who to hire and fire, how much to pay people? Who disciplines the bad people and encourages the good ones?”

Kate smiled. “Good questions, and questions that Scrum really does not want to answer. Yes, in a real organization, there will be some connection to personnel, to some kind of technical management, and so on. Scrum, to me, is defined as if those things were not needed, or as if the team could do all those things. What’s interesting, of course, is that in a startup situation, the team really can do all those things. What was it like when you started Oak River Software?”

Dan thought back. “Well, I made some final decisions about hiring and firing. Certainly I set the budget, because it was my money. But mostly, Art Givens, who was my marketing and domain guy back then, decided what we needed to do. And I had a lead developer who decided who to hire and fire, within budget. I’d counsel with him when there were issues, and I often had to get involved because his people skills weren’t the best. But we didn’t do a lot of what I’d really call management. We worked together, figured out what had to be done, and did it.”

Dan paused. “You know, maybe this is the entrepreneur in me talking, but I think things were great back then. It was only later, when Art left, and I had to hire marketing and sales managers, and a real technical manager to drive the programmers, when things started to go bad. I supposed it was just part of how a startup company becomes a grownup company. Honestly, I liked the old days best.”

Kate said, “Let’s get back to Scrum. We’ll leave it that Scrum doesn’t talk directly about conventional management, and that it sounds a lot like what a team-focused startup might do. We can come back to what larger companies need to do after we get the full picture.

“Let me say a few more words about the roles. First, the team. Since the team is responsible for doing what the Product Owner asks for, the team needs to have all the skills to make that happen. It wouldn’t make sense, for example, to have a software team without testing. We couldn’t rightly hold them responsible for producing working tested features if the testing was done separately.”

“Wait a minute, Kate,” Dan said. “Everyone knows that developers don’t make good testers.”

“You might be surprised,” said Kate. “But the team isn’t just developers. The team in Scrum must have all the skills necessary to produce an increment of potentially shippable software in every Sprint. That means everything.”

“What if the product needs a manual,” said Dan. “Do they have to write a manual every Sprint, and keep it up to date?”

“In a word, yes,” Kate said. “The Product Owner determines what kind of documentation the product needs, and the team is responsible for producing it. And there are Scrum teams all over that include tech writers, who produce a draft of the manual every Sprint, just like the developers and testers produce a draft of the software.”

“How is that even possible,” Dan said. “They’d have to revise the manual every Sprint.”

“Yes, they would. However, the devs and testers have to revise the actual software every Sprint. Their stuff has to be usable, actually work, get correct answers, and so on. That’s actually harder to do than writing a manual. If a programmer leaves out a semicolon, the program won’t work. If a writer leaves one out, the document still works rather well.

“The proof is in the doing. As I say, there are teams all over who do this, producing more and more refined documentation as the team produces more and more refined software.”

Dan looked confused. “What if the Product Owner changes her mind about what the product has to do? The writers would have to change everything.”

Kate said, “If the PO changes her mind in some major way it is a big deal. Frankly it’s a bigger deal for the developers and testers than it can ever be for the writers. Anyway, Scrum requires the PO to decide what the product is and how to deliver working bits of it over a number of Sprints, and requires that the team be cross-functional enough to do what is needed.”

“What about resources,” said Dan. “Isn’t it wasteful, putting a documentation person on every team, and so on?”

“The team can change over time, depending on need. What is really wasteful is giving the team half a writer and two-thirds of a tester and seven sixteenths of an analyst. Scrum asks that we make some hard decisions about how to staff our projects, and our organization. Frankly, I think that’s a strength of the method: it requires us to make real decisions, not load-sharing everyone gives up fifteen percent kinds of decisions.”

“OK,” Dan said. “I think I’m getting the idea. You give the team everything they need, including someone who can actually decide what needs to be done, and set them loose with a budget. Seems like anyone could succeed if they got everything they needed.”

“Exactly!” Kate said. “Scrum starts with everything you need. A wise project creator would fund his most important projects fully, then work down the list until he ran out of people or money. Let’s face it, starving all our projects can’t possibly be a good idea. Anyway, let’s move on.

“Scrum relies on a fundamental notion, ‘Inspect and Adapt’. The whole team inspects what happens, using day-to-day events, and Sprint-to-Sprint events, to see clearly how things are going. When they see opportunities to improve, they put improvements in place. As we’ll talk about later, this is one of the places where Scrum can fail or go poorly: if you do not observe problems, or do not or cannot fix them, you can’t possibly prosper. It’s surprising how many organizations seem to forget this, instead just telling the team to soldier on.”

Kate made a note on a card: “How Scrum Fails,” then went on.

“Scrum defines four meetings. Each of these offers opportunities to Inspect and Adapt. They all fit within the big time box, the Sprint. Ideally, I think a Sprint should be no more than a couple of weeks long.

“Anyway, the meetings. First, there’s the Sprint Planning Meeting. At the beginning of each Sprint, everyone gets together and works out what will be accomplished in the upcoming Sprint. The PO provides ‘backlog items’, the things she wants, in the order she prefers them. The team considers these, and the technical means of accomplishing them. There can be some give and take, and at the end of the meeting, the team all understand what will be undertaken, and how they plan to accomplish it.

“Note that this is a conversation. A negotiation, if you will, but I prefer to think it’s a conversation with everyone on the same side. The team can’t do more than they can do, and it would be counter-productive if the PO were to demand more. Two weeks later, she’d get what they could do and frankly if the PO and team continue to forecast more than is accomplished, it’s evidence of undue and unrealistic pressure from somewhere.

“Again, at the end of this meeting, which should take a morning for a two-week Sprint, everyone on the team knows what’s going to be undertaken, why it’s the thing to do, and how they plan to get it done. Capisce?”

Dan said, “I think so. It seems simple enough. Every couple of weeks, they all get together, look at what the PO needs next, and figure out how much they can do and how to do it.”

Kate said, “Right. Now let’s look at the other end of the Sprint. The Sprint has two meetings at the end. The first is the ‘Sprint Demo’ or ‘Sprint Review’. In this session, the team demonstrates to the Product Owner what they have accomplished. Everyone discusses what happened, what issues were encountered and resolved, and so on. They do a little looking into the future, based on a fresh understanding of what has just been done, preparing for the next Sprint Planning Meeting. Here, we are inspecting the product and adapting our expectations of what is to come.

“The second meeting is the Sprint Retrospective. Here, we look at how we worked as a team and how our process and tools supported us. We brainstorm ways to improve, and agree to specific actions that we will take in the next Sprint to improve things. Often these changes are inside the team. Sometimes, they require someone outside to take some action, in which case we plan how we’ll influence them to do what’s needed. This meeting is an inspection of the process itself, and a determination about how to adapt to improve it.

“All three of these meetings are mandatory in Scrum. It would be extremely unwise to drop them or give them short shrift. Are you with me so far?”

Dan said, “Let me refresh the drinks and get my thoughts in order.”

“Sure,” said Kate.

Dan returned with drinks and a summary. “OK, so Scrum has these Sprints, a couple of weeks long, where the Product Owner and the team get together to plan what has to be done and how, then to see how well it went and figure out how to improve. The team has all the abilities they need, and they produce a bit of product every Sprint. Right? But you said there were four meetings, didn’t you?”

Kate looked happy. “Yes, I think you’ve got it so far. And the fourth meeting occurs daily. It’s called the “Daily Scrum’, I guess after some ritual that takes place in a sports event. It’s very simple and very brief: in fact, usually, it is done standing up, although I choose not to do that.

“Basically, every team member says what they accomplished yesterday, what they will accomplish before the next Scrum, and what, if anything, is standing in the way. They don’t try to solve the issues during this meeting, but if there are issues, most teams meet together or in small groups to address them right after the Daily Scrum. So what are the Scrum meetings?”

Dan said, “No one told me there would be a kind of inquisition.”

“No one expects the inquisition but I’m sure you’ve got it,” said Kate. “Go on.”

Dan said, “OK, there’s the planning meeting, the daily meeting, the demonstration and the — what did you call it, retrospective.”

“By Jove, I think he’s got it. What else is there, Dan?”

“You’re asking me? I don’t know. Do they do anything besides meet? Do they build anything, write reports, or the like?”

Kate said, “Yes. Yes, they do. There are what we call ‘Scrum Artifacts’. The most important of these is what Scrum calls the ‘Increment’, and what I call ‘What have you done for me lately’.

“At the end of each Sprint, we demonstrate an increment of software. This software is required to be ‘done’, by our then-current ‘Definition of Done’. The definition might start out simple, such as ‘able to be demonstrated on the development system’. But over the course of a good project, the Definition of Done itself gets more and more stringent. ‘Demonstrated on the PO’s system’, ‘documented and demonstrated on the production servers’, and so on. Some teams actually call things done, every two weeks, only when a customer has actually paid money to use it. That’s impressive, and Oak River isn’t there yet. Our definition of done is pretty strict however. Let me quote it:

“An increment is done when all the PO’s defined acceptance tests run, when the system is fully integrated and running on the production servers, when all the manual pages are up to date, the code is clean and well-factored, and all the programmer tests are running green.

“Our Product Owners do not accept anything that doesn’t meet those criteria, even the technical ones. They don’t inspect those criteria: the development team certifies that those criteria are met. But we are all serious about meeting them, and we’ll call a feature ‘Not Done’ before we’ll slack on them.”

“Wait,” Dan said. “Are’t there times when you’d prefer to get a feature out without clean code or some such techie concern?”

Kate said, “You’d think so, at first. Emergencies arise and so on. Truth is, if a team can release what looks like a feature faster by writing poor code, that’s major evidence that they have important things to learn about how to write software. A good professional produces software at the best pace by working clean.

“This means that every time we feel we ‘need’ to push something out that isn’t really ‘Done’, we inspect what’s going on and decide how to improve. So if we do have to rush out a defect fix, we are all committed just as much to fixing our process to make that unnecessary. It doesn’t take long until we are able to build increments that meet our Definition of Done fully, nearly every time. And if we do stumble, we go right back to inspect and adapt and work to make sure it won’t happen again.”

“That seems incredibly idealistic,” Dan said. “Things never go that well. Even if you tried, it would cost a fortune to be that good.”

“They can, Dan, and they do. In the early days of manufacturing, products were built, inspected, and sent back to be redone if things were wrong. Manufacturing people learned that inspecting earlier and more often meant that fewer products had to go back to be redone. Small inspections and small fixes to process are immensely less costly than reworking the whole product item.

“Similarly for software. Debugging a fully integrated system is difficult and costly, especially if it is made up of components that are not all individually nearly bug free. So we save time and money by testing at every level. We have automated checks that things work, and we run them all the time, and add to them all the time. The result is that things almost always go well, and we get very clear indications of where we’ve messed up when we do.”

Dan said, “But I have been reading Eric Ries book, The Lean Startup. He makes the point that it can be best to go out with far less product, even with a facade, rather than build out for perfection. You learn faster.”

Kate said, “Yes. But the thin character of such a product should be chosen. When we write code badly, we don’t get to choose where the problems are: they show up where we least expect them and where we don’t want them. So even when we’re building a minimal product experiment, we do best to work clean. We’re not gold-plating, but we are working clean. Anyway, from the Scrum viewpoint, this is all part of the Definition of Done. If a PO wanted the Definition of Done to include “code is the most rag-tag crappy code that will barely carry the load”, that could be the definition. Scrum doesn’t care. I care, and I wouldn’t work that way. But Scrum doesn’t care.

“But we’re talking about the artifacts. The main one is the increment. Without the increment, we have no product. But as the main says, ‘Wait, there’s more’.”

Kate went on. “There’s the Product Backlog. That’s just the Product Owner’s current list of things that she might want in the product someday. It’s kept ‘ordered’, generally speaking in the order that the PO would like to have things done in. ‘Order’ can consider a lot of concerns: how much we need to learn whatever that item will teach us; how much revenue we’ll get for it; how much pressure some stakeholder is putting on us to show him some results; and so on. What matters is that the PO and team considers and defines the order of the items.”

Kate took a breath, to the delight of observers near and far. “The Product Owner monitors progress in completing backlog items. She uses whatever means she finds useful, from comparing stacks of done and not-done cards, to charts showing the growth of features, even to Earned Value kinds of calculations if they’re into that sort of thing. She and the team continually ‘Groom’ the backlog, refining items, adding new items, removing items that no longer seem important. With me so far?”

Dan, recovering, said, “Yes. The Product Backlog is all the stuff that needs to be done, or might need to be done. I suppose items soon to be done are well fleshed-out …” He coughed, then went on “better understood and defined than things that are further down in the order.”

Kate feigned confusion but couldn’t suppress a grin. “Consider it a little treat. Anyway you’re right. Then there’s the Sprint Backlog. That’s just the list of things agreed to be done in the current Sprint. and the plan for getting them done. Just as the PO monitors overall progress in terms of the backlog, the team monitors progress toward attaining the Sprint goal. And, again, this can be done in any number of ways. My favorite is to have all the items on cards, and show them moving through whatever work stages they go through, so that I can see how many cards have arrived at the ‘Done’ station every day.”

Kate continued, “And that’s Scrum. Three roles, Product Owner, Scrum Master, and Team. Four meetings, Sprint Planning, Daily Scrum, Sprint Demo, Sprint Retrospective. And three artifacts, Product Backlog, Sprint Backlog, and the Product Increment. What have we missed?”

Dan thought a moment. “You haven’t said much about the Scrum Master, just that he facilitates and helps the team improve.”

Kate said, “And that’s about it. It seems like a simple job, and in sufficiently effective teams, it might even be absorbed into the team’s general operation. But truth is, there’s a lot to do and a good Scrum Master is priceless. Let me count some of the ways:

“The Scrum Master helps the PO do her job. He helps everyone understand how to create good backlog items, how to communicate them, how to plan an adjust in as dynamic an environment as most projects are, how to do all the things Agile teams need to do, and so on. He helps the team do their job, coaching, leading, even teaching them how to do things. He ensures that impediments are removed. He serves the organization, helping them understand Scrum, their role in it, and the impact of the things they do on the success of the Scrum team or teams. He works throughout the organization to help Scrum work. Best of all, he does this with no formal management authority.”

Dan said, “Sounds like a hard job.”

“It is,” Kate said. “Especially when you consider that you can become a Certified Scrum Master with two days of training. The range of things that may need to be done to help a team become really good, and the breadth of the organizational reach involved, can be immense. Scrum Master is either a very hard job, or perhaps not a person’s job at all. All those things need to be done, so doing them is surely a ‘role’. But whether that work can be done by a single person, I seriously doubt. Whether a person can be trained to do them … well, I don’t doubt. I know they can’t be. Doing everything that it takes to make a Scrum team effective requires so many skills and so much skill, that it’s a lifetime’s work.”

“But wait,” Dan said. “Are you saying that Scrum basically can’t work, because the Scrum Master role is too much for a single person?”

“Not at all,” Kate said. “Scrum can and does work just fine. Most Scrum teams get at least a reasonable amount of benefit, and teams that pay attention to removing their impediments can absolutely fly. Our teams are beginning to show that now. What needs to happen is that the team must pay attention to the Scrum Master role, and the company must pay attention to it as well. There needs to be continuing attention on improving the effectiveness of that role, and there needs to be recognition that sometimes parts of the Scrum Master role need to be done by other team members, or even people outside the team.

“So there’s your quick overview of Scrum. There’s much more we should talk about before you decide what to do with these new people you’re looking at. I’d suggest we get together again over the next few days to talk about how Scrum and Scrum teams can fail, about whether we at Oak River are actually doing Scrum, about other Agile approaches that are not like Scrum, about all the important technical and related practices that need to be in place, and more.”

“Thanks,” Dan said. “I think I’ve heard about all I can absorb for today. Let me check my schedule for the test of the week and drop you an email proposing another meeting. We can do it at the office if you like, of course.”

Kate reflected for a moment. “We might do that. There might be some things to talk about where some other people might be helpful. Otherwise, I like meeting here. The buzz of activity is energizing. Either way, let me know when you’re available and we’ll pick a time and place, and I’ll see you then.”

“Thanks, Kate,” Dan said. “I’ll just clear the table.”

When Dan returned from the trash bin, Kate was gone. Not even her smile remained.

The Scrum Alliance was the main supporter of the Agile Skills Project, and we’ll certainly be building on that model. We have also been behind the Alliance’s recent inviting of articles from Agile- / Lean- / Craft-focused experts. This is just a beginning.

The Agile Skills Project identified seven “pillars” of skill: Business Value, Collaboration, Confidence, Product, Self Improvement, Supportive Culture, and Technical Excellence. We’re not sure these are the right seven, or even whether seven is the right number. What we are sure of is that there are many many skills that organizations, teams, and individuals can bring to bear on making Scrum and Agile projects successful. We hope to provide an overview of these, a roadmap through them, detailed information within this project, and pointers to outside sources and experts.

The scope of what we “might” do is of course immense. We will be doing our best to keep our backlog public, as well as our criteria for ordering it. We will be clear about which items are being worked on, and which ones are not. Our purpose will be to deliver value quickly, and to build a space within which growth can continue indefinitely, as our community understanding grows and changes.

As long-term Agilists, XPers, and Certified Scrum Trainers, plus a bit of Certified Curmudgeon, we will be trying to bring a combination of independence and Scrum expertise to this enterprise. If you think we lack something in independence or expertise, we sincerely solicit your help. We would like this growing information resource to be of real value to the “world of work”. As such, if you would like to help by writing, by editing, by commenting, or in any other way, we welcome your help. We see ourselves as curators and editors, not as prophets or definers of what is good and what is bad.

We’re just getting started. We need to create our backlog, get it published somewhere, and to begin to find our way through the items, figuring out how to get them done.  As soon as possible, we’ll be getting sites set up where you can help. We might use the Agile Skills Project site, or someplace new. Meanwhile you can write to me at ronjeffries at acm dot org, or comment here. We’ll be much more grateful for concrete help, but welcome all advice and positive-going ideas.

Again: we’re just getting started. Right now, it’s just us, and we have about a week a month to put into the effort. We’ll try our best to be open and transparent and to be delivering value rapidly. Call us on it if we’re not. And help us if you can.

Thanks!

At Agile2011, I brought along a “gift”, a nicely formatted and illustrated Kate Oneal story. I gave a copy to everyone who asked for one, and to a few people who didn’t but who I wanted to have one anyway.

This was too clever by half. I made two mistakes, so perhaps it was even too clever by two-thirds. First, I tweeted about the thing only with no hashtag or #gotKate. I should have tweeted on #Agile2011. I momentarily forgot that not everyone in the world follows me.

Second, I made it too hard to get one. People don’t like to ask for things from people they don’t know well. Some people may have just stood near me hoping that I’d give them one — I’ll never know. Some people would come up and the conversation would go like this:

Some One said, “Hi Ron.”

“Hi Some,” Ron said.

“So, I hear you have some kind of a gift for people?”

“Yes, I have.”

“Is it a nice gift?”

“I like to think so”

This would go on at some length until Some One either figured out to say “May I have one?” or until Chet, bored with the game, would say “Ask him for one.” Once in a while I’d say “Would you like one,” but too often I was stuck in the nerd game of just answering the question logically but not usefully.

That was, not to put too fine a point on it, stupid, if my point was to give away as many of the things as possible.

I did manage to give away all that I was carrying with me every day, but never so many that I had to go back to my room and replenish early. So I’m sure there could have been more given out.

You Can Get One

Speaking of that, I’m willing to send a signed one to anyone who wants it, and who sends me a self-addressed stamped envelope. I will weigh the thing to see how much postage it needs, but two-ounce postage will surely do it. The booklet is 5.5 by 5 inches and you might want a stiff or padded envelope so that it doesn’t get crunched. I’ll update this paragraph when I know more.

If you have the thing and like it and would like a bunch to hand out, drop me an email and we’ll see what we can set up.

Feedback Please

Many of the people who got them expressed later that they liked them, so that’s good. Here, I’m soliciting more feedback on whether you liked it, whether you tried to get one and I never realized it, and on what should be done next.

I’ll tell the story of how we made the thing in another posting. In this one, I’d just like comments, offering ideas about what the thing is, how you liked it, whether I should do another one, whether you can think of some way to monetize the creation of these, and so on. Anything on the topic, really.

Thanks!

Just discovered your blog and am enjoying it. I was struck however bythe fact that your test code would not pass muster in my team. We have more test code than non-test code, so we are equally rigorous about the quality of the test case as we are the non-test code (quality insomuch as it is likely to affect the future costs of owning the code). Therefore I consider repeating yourself in test code to be almost invariably bad practice. Yet your test code is sprinkled with repeated literals. You also have subtle repetition – a test method named ..sixth… which includes six calls to member.charge.

I’m sure that you have a good reason and I wonder what it is.

I do have some good reasons … some that are questionable, and some that are probably not so good at all. Let’s explore the ones I’m aware of. My tabbed out comments are pretty much what I said to Piers. Tabbed in are today’s feelings about that day’s feelings.

First of all, the article series so far is an example of TDDing to a point early in a program. And in fact some of the things, like the literals, do get pulled out later on. The TDD cycle is red/green/refactor and one improves what one sees as one goes.

True enough: the focus is on getting a non-DB solution. However, there are a pile of literal small integers and they do not have meaning. I did pull out the person ones (not as nicely as I might have) but the value ones are not yet pulled out. They should be, and in my current version they are. I’ll post that later.

Recall also that this is my first-ever published Python program. AsI comment in the second article, I was doing things (like not adhering to “standard” naming) to reduce my cognitive load. Possiblythat same focus on “how the hell do I say this in Python” caused me not to notice things that I otherwise would have.

And, I didn’t have anyone pairing with me to help keep me honest.

As an explanation of how I got where I got, I buy this. I wouldn’t accept it as an excuse for being there, or for staying there. The code is not good enough for the long term, and even with a few days’ experience in Python, I can see that. I don’t know the Python way of making it better but I sure as hell know some ways. Again, let’s see what the next version looks like.

However, I actually do some of these things on purpose.

In the method Piers mentions, the six calls are necessary, as I’m sure you see, because I want to check that each of the six gets a correct answer, including the sixth, which is supposed to get zero. This would have been a better test, perhaps, if all the values had been different, but since TDD is a white-box activity, I wasn’t worried about getting some number other than the input or zero. It /might/ improve the tests for the long term to change that.

The six calls could, of course, be placed in a loop, as I did with subsequent ones. There are two reasons not to do that, one not so good and the other one more to the idea of “reasons” why I do what I do.

The not so good reason was that at that moment I didn’t want to figure out the looping and how to construct it. I had “call six times” in my mind and writing it out longhand was easier.

The better reason, however, is that doing the calls explicitly, rather than in a loop, tells a better story. The member buys something; he buys something later; he buys something later still; and so on. The “real world” events are not “the member is in a loop”, so I prefer the code written out longhand, despite the duplication.

However, in the later, more complex tests, with multiple members making multiple purchases, I did go to the looping, and frankly I think it makes the tests harder to understand. I could imagine some heavier refactoring, such as making

    for i in range(5):
        self.assertEquals(6,member.charge(6))

into something like

    assert_member_makes_five_charges_at_cost(6)

And one could make that (6) into (SIX_DOLLARS), and probably should.

Suffice to say that I like each test to tell its own story, and so I generally do not factor out duplication as aggressively as I would in the production code. This is an example of the tension between rules two and three of  Beck’s rules of Simple Code:

1. Runs all the tests;
2. Contains no duplication;
3. Expresses all our design ideas;
4. Minimizes number of code entities;

In the case of tests, I sometimes prefer more duplication to get better expression of the testing ideas.

For the article, this final point is also key:

The point of the article is to focus on how to TDD toward a database without actually doing any database work. Thus I want the code in the article to focus on the working mechanics of the production code, and on the thought process that goes into the tests. If I were to go off on a refactoring spree on the tests, I felt that would draw attention away from the core point.

That said, for longer-term life in a production system, the tests do need improvement. Here’s what I have for those same tests today:

THREE_DOLLARS = 3
FOUR_DOLLARS = 4
FIVE_DOLLARS = 5
SIX_DOLLARS = 6
TEN_DOLLARS = 10

PERSON_ONE = 1
PERSON_TWO = 2
PERSON_SIX = 6

class NoDBTests(unittest.TestCase):

    def test_hookup(self):
        self.assertTrue(True)

    def test_member_charge(self):
        member = MemberRecord()
        charge_amount = member.charge(TEN_DOLLARS)
        self.assertEqual(TEN_DOLLARS,charge_amount)

    def test_sixth_one_free(self):
        member = MemberRecord()
        charge_amount = member.charge(FIVE_DOLLARS)
        self.assertEqual(FIVE_DOLLARS, charge_amount)
        charge_amount = member.charge(FIVE_DOLLARS)
        self.assertEqual(FIVE_DOLLARS, charge_amount)
        charge_amount = member.charge(FIVE_DOLLARS)
        self.assertEqual(FIVE_DOLLARS, charge_amount)
        charge_amount = member.charge(FIVE_DOLLARS)
        self.assertEqual(FIVE_DOLLARS, charge_amount)
        charge_amount = member.charge(FIVE_DOLLARS)
        self.assertEqual(FIVE_DOLLARS, charge_amount)
        charge_amount = member.charge(FIVE_DOLLARS)
        self.assertEqual(0, charge_amount)

    def test_twelfth_one_free(self):
        member = MemberRecord()
        for i in range(5):
            self.assertEquals(SIX_DOLLARS,member.charge(SIX_DOLLARS))
        charge_amount = member.charge(SIX_DOLLARS)
        self.assertEqual(0, charge_amount, "first one bad")
        for i in range(5):
            self.assertEquals(SIX_DOLLARS,member.charge(SIX_DOLLARS))
        charge_amount = member.charge(SIX_DOLLARS)
        self.assertEquals(0, charge_amount, "second one bad")

    def test_member_two_still_gets_free(self):
        members = MemberCollection()
        for purchase_from_two in range(5):
            self.assertEquals(FOUR_DOLLARS, members.member(PERSON_TWO).charge(FOUR_DOLLARS))
        self.assertEquals(0, members.member(PERSON_TWO).charge(FOUR_DOLLARS))

    def test_member_six_does_not_interfere_with_two(self):
        members = MemberCollection()
        for purchase_from_two in range(5):
            self.assertEquals(FOUR_DOLLARS, members.member(PERSON_TWO).charge(FOUR_DOLLARS))
        self.assertEquals(FOUR_DOLLARS, members.member(PERSON_SIX).charge(FOUR_DOLLARS), "six interfered")
        self.assertEquals(0, members.member(PERSON_TWO).charge(FOUR_DOLLARS), "two got no discount")

    def test_threeAtOnce(self):
        members = MemberCollection()
        for purchase_number in range(5):
            for person in [2, 6, 1]:
                self.assertEquals(THREE_DOLLARS, members.member(person).charge(THREE_DOLLARS), "improper free one")
        self.assertEqual(0, members.member(PERSON_ONE).charge(THREE_DOLLARS))
        self.assertEqual(0, members.member(PERSON_TWO).charge(THREE_DOLLARS))
        self.assertEqual(0, members.member(PERSON_SIX).charge(THREE_DOLLARS))

 
Of these, I like the named literals much better than the literal literals. This was Piers’ first point. As for the duplicated lines versus the loops, I find the tests with the duplicated lines easier to read by a small margin. The last one, looping over five interleaved purchases for three people, is pretty hard to decode, I think. I prefer the six in a row one to a single loop of five followed by a single call to check the freebie, but it’s hard to count the six. If it were “twelfth one free”, in line would be bad for sure.

So let’s see what else we can do. First of all, all those asserts have a log of duplication, and they’re all just checking that some person pays some amount, or that some person pays nothing. Let’s express that:

    def check_person_pays(self, members, person, amount):
        self.assertEquals(amount, members.member(person).charge(amount))

    def check_person_free(self, members, person, amount):
        self.assertEquals(0, members.member(person).charge(amount))

    def test_member_two_still_gets_free(self):
        members = MemberCollection()
        for purchase_from_two in range(5):
            self.check_person_pays(members, PERSON_TWO, FOUR_DOLLARS)
        self.check_person_free(members, PERSON_TWO, FOUR_DOLLARS)

    def test_member_six_does_not_interfere_with_two(self):
        members = MemberCollection()
        for purchase_from_two in range(5):
            self.check_person_pays(members, PERSON_TWO, FOUR_DOLLARS)
        self.check_person_pays(members, PERSON_SIX, FOUR_DOLLARS)
        self.check_person_free(members, PERSON_TWO, FOUR_DOLLARS)

    def test_threeAtOnce(self):
        members = MemberCollection()
        for purchase_number in range(5):
            for person in [2, 6, 1]:
                self.check_person_pays(members, person, THREE_DOLLARS)
        self.check_person_free(members, PERSON_ONE, THREE_DOLLARS)
        self.check_person_free(members, PERSON_TWO, THREE_DOLLARS)
        self.check_person_free(members, PERSON_SIX, THREE_DOLLARS)

I think I like that better still. We have lost my few special diagnostics, but those can readily be replaced by adding an optional argument to the check functions. Meanwhile the tests express what’s going on quite a bit better.

I think this is well worth doing before committing these stories to the system. And I’m still glad that we focused first on the main point, getting the database feature working without an actual database, and then looked at improving the tests.

What do you think, Piers? And what do you all think? Thanks!

First, the Tests

Nothing new to see here. These are in the order they were written, which is my usual practice with new code. If a test suite gets large, I’d break it up, and perhaps arrange tests by topic area. As a rule, though, I like them to tell the story of how the program grew.

import unittest
from membercollection import MemberCollection
from memberrecord import MemberRecord

class NoDBTests(unittest.TestCase):

    personOne = 1
    personTwo = 2
    personSix = 6

    def test_hookup(self):
        self.assertTrue(True)

    def test_memberCharge(self):
        member = MemberRecord()
        chargeAmount = member.charge(10)
        self.assertEqual(10,chargeAmount)

    def test_sixthOneFree(self):
        member = MemberRecord()
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(0, chargeAmount)

    def test_twelfthOneFree(self):
        member = MemberRecord()
        for i in range(5):
            self.assertEquals(6,member.charge(6))
        chargeAmount = member.charge(6)
        self.assertEqual(0, chargeAmount, "first one bad")
        for i in range(5):
            self.assertEquals(6,member.charge(6))
        chargeAmount = member.charge(6)
        self.assertEqual(0, chargeAmount, "second one bad")

    def test_memberTwoStillGetsFree(self):
        members = MemberCollection()
        for purchaseFromTwo in range(5):
            self.assertEquals(4, members.member(self.personTwo).charge(4))
        self.assertEquals(0, members.member(self.personTwo).charge(4))

    def test_memberSixDoesNotInterfereWithTwo(self):
        members = MemberCollection()
        for purchaseFromTwo in range(5):
            self.assertEquals(4, members.member(self.personTwo).charge(4))
        self.assertEquals(4, members.member(self.personSix).charge(4), "six interfered")
        self.assertEquals(0, members.member(self.personTwo).charge(4), "two got no discount")

    def test_threeAtOnce(self):
        members = MemberCollection()
        for purchaseNumber in range(5):
            for person in [2, 6, 1]:
                self.assertEquals(3, members.member(person).charge(3), "improper free one")
        self.assertEqual(0, members.member(self.personOne).charge(3))
        self.assertEqual(0, members.member(self.personTwo).charge(3))
        self.assertEqual(0, members.member(self.personSix).charge(3))

if __name__ == '__main__':
    unittest.main()

The MemberRecord

This, too, is pretty much as we’ve seen it. It may be worth mentioning that the member variable __purchaseCount is private, by virtue of the two leading underbars, and that memberNumber is public. We wanted it public so that our collection code could access it.

As far as I know, there is no access indicator for read but not write. If I really cared, I’d build an accessor method and use a private variable. However, I’m usually not very concerned that someone might mistakenly write into a member record. I’m used to working with teams who have an agreed approach to their code. If I were writing a library for use by outsiders, I might be more paranoid.

__author__ = 'Ron'
class MemberRecord(object):

    def __init__(self, memberNumber=-1):
        self.__purchaseCount = 0
        self.memberNumber = memberNumber

    def charge(self, amount):
        self.__purchaseCount+=1
        return amount if self.__purchaseCount%6 != 0 else 0

The MemberCollection

Here, I changed the way the member(memberNumber) function works, using an explicit loop rather than my overly clever approach of filtering and then checking the result size. Note, however, that the explicit loop would be a Very Bad Idea if we were to translate it directly into a database solution. We wouldn’t want to read the database one record at a time. The filter approach might better suggest what to do to someone building the  database code. Again, my normal approach is to trust the team. YMMV.

from memberrecord import MemberRecord

__author__ = 'Ron'
class MemberCollection(object):

    def __init__(self):
        self.members = []

    def member(self, memberNumber):
        for mr in self.members:
            if mr.memberNumber == memberNumber:
                return mr
        newMember = MemberRecord(memberNumber)
        self.members.append(newMember)
        return newMember

Better Idea

Ori Peleg commented on the earlier article that there is a better way to do my member finding method, using a dictionary. When I worked on another practice program, I had more than one entry with the same (partial) key, so needed to filter or loop. Somehow that stuck in my mind and I didn’t think to use a dictionary, though I had tried one in the other location.

I blame my pair, Chet, for not being there. Anyway, the better way is:

 def member(self, memberNumber):
        if memberNumber in self.members:
            return self.members[memberNumber]
        else:
            newMember = MemberRecord(memberNumber)
            self.members[memberNumber] = newMember
            return newMember

Comments on Python Style

The code here is not in official Python style. In particular, it seems that Pythonistas prefer send_player_to_jail over sendPlayerToJail. That said, the guide linked to above does allow for use of mixedCase where that is the prevailing style.

One thing you’d have to think about if adding Python to the development mix on your team is what style to follow. At the time I wrote this program, I knew that underbar_names were preferred, but because I’m used to mixedCase, I used it to keep my cognitive load down.  I was sure someone would sort me out on it, and my luck was in. Oddly enough, Python folks do use CapWords for class names but not mixedCase for methods. Curious.

There may be other deviations: the guide referenced above is long and detailed. If I publish more Python code, I’ll see about coming more in line with convention. Meanwhile, I’ll refactor this program just to see how easy PyCharm makes it, and to bring it more in line. I won’t trouble you with another version unless I feel the need for more feedback.

Thanks for your help and comments, I do appreciate them.

Bottom Line … For Now

There are probably more things we could change here, either to make the code look more like good Python, or to make it more clear, or for some other reason. Since my point in the previous article was just to show how I’d proceed to solve a database problem without involving the database yet, I think the point is made.

Your comments are welcome …

Hmm. First we have to think of an app. I don’t want to do the one we use in the class: that would make it too easy for the next vict^H^H^H^H lucky participants.  I’m at the Seattle’s Best coffee shop in Borders, so how about this:

Whenever someone buys something in our coffee shop, we remember how many things they have bought, and every time they buy another five things, the sixth one is free. People and their purchases are recorded by their membership number in our Constant Caffeination Club.

Clearly this requires a database. Surely we will have millions of members in the CC Club as soon as the word gets out about this fabulous deal. We can’t possibly build this system in a TDD style without using some kind of database software.

To make this even more difficult, we’ll build the software in Python, since I just got Python and haven’t much of an idea how to program in it. Fortunately, I also have a trial of JetBrains’ PyCharm, which is very helpful. I’ll be buying that before the week is out, I think. Naturally, I begin with a hookup test:

import unittest

class NoDBTests(unittest.TestCase):
    def test_hookup(self):
        self.assertTrue(True)

if __name__ == '__main__':
    unittest.main()

With great good fortune, this goes green. Now let’s TDD a class into existence. I’m thinking I’ll start with the Club member record, and … oh, let’s see … the way it works is that we get the member record and when they buy something for X dollars, we send the member record a message and it responds with the amount to charge the member … either X or zero, probably. This will take a couple of tests, or one growing test. Let’s see:

    def test_memberCharge(self):
        member = MemberRecord()
        chargeAmount = member.charge(10)
        self.assertEqual(10,chargeAmount)

This ought to serve to get us going. PyCharm tells me there is no MemberRecord class and offers to build it for me. I allow that. It builds it into the current file. I can live with that for now. I expect that when I run the tests it will go red-bar on charge not existing … and it does. I will add that, as follows:

class MemberRecord(object):
    def charge(self, amount):
        return 0

This gets me the next expected red-bar, namely 10 != 0, telling me that my implementation of charge() is wrong. Naturally, I fix it perfectly like this:

class MemberRecord(object):
    def charge(self, amount):
        return 10

OK, there can be no doubt that this is the definitive implementation of charge. So it is time for another test. Let’s have our guy charge six things and check that the first five come back charging, and the sixth comes back free. I’m sure this is going to work.

    def test_sixthOneFree(self):
        member = MemberRecord()
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(5, chargeAmount)
        chargeAmount = member.charge(5)
        self.assertEqual(0, chargeAmount)

Damn. Didn’t quite work. Turns out returning 10 was wrong. I need to return the amount sent in. Easy fix:

class MemberRecord(object):
    def charge(self, amount):
        return amount

Darn. (Not quite as bad as damn.) The first five asserts worked fine but the sixth one returned five instead of zero. What’s up with that?

N.B. I’m not really this stupid. I’m just having fun. Really. You can trust me on this.

Anyway, we need to implement the counting from one to five to make this work. Should be easy enough:

class MemberRecord(object):

    def __init__(self):
        self.__purchaseCount = 0

    def charge(self, amount):
        self.__purchaseCount+=1
        return amount if self.__purchaseCount != 6 else 0

Pretty big change there, but let me point out that it was mostly addition, not rewriting.

Now of course, I was just doing the absolute simplest thing there, and it is clear that a test of more than six purchases will fail to give any more freebies. Let’s try 12:

    def test_twelvthOneFree(self):
        member = MemberRecord()
        for i in range(5):
            member.charge(6)
        chargeAmount = member.charge(6)
        self.assertEqual(0, chargeAmount, "first one bad")
        for i in range(5):
            member.charge(6)
        chargeAmount = member.charge(6)
        self.assertEqual(0, chargeAmount, "second one bad")

I did this in two steps to be sure that the first one still works. Sure enough, the test fails saying “second one bad”. I’m thinking I have a fix for that … I’ll do a mod check on the count. I could zero it but I think the mod is easier and a bit more flexible. I understand mod, so others might disagree. Anyway here goes:

   def charge(self, amount):
        self.__purchaseCount+=1
        return amount if self.__purchaseCount%6 != 0 else 0

As I thought, this works just fine. We are green all the way, and I’m sure that the system is working just fine. Might be a good idea to beef up this final test just a bit, to make sure that it’s not always returning zero for some reason. Like this:

    def test_twelvthOneFree(self):
        member = MemberRecord()
        for i in range(5):
            self.assertEquals(6,member.charge(6))
        chargeAmount = member.charge(6)
        self.assertEqual(0, chargeAmount, "first one bad")
        for i in range(5):
            self.assertEquals(6,member.charge(6))
        chargeAmount = member.charge(6)
        self.assertEqual(0, chargeAmount, "second one bad")

Still green, and a bit more robust. Naturally, you could have gone faster, but you’re not writing an article on how to do TDD, and you probably don’t like going down to the finest grain possible. I do. One result of that is that never once in this bit of coding did I get a surprise. (Yes, I was faking it. I wanted you to think I was having a good time.)

I like working with no surprises. It gives me confidence that I’m coding what I set out to code. When I do get  a surprise, I try to back out my last changes, and go in smaller steps. When I forget to do that, and start debugging instead, pretty soon I’m strewing print statements around or even going into the debugger. So far I don’t know how to work the PyCharm debugger and I’m hoping never to learn.

Hey! I thought this was supposed to be about databases???

Hey, yourself. Try to pay closer attention. This is totally about databases.

Why is this even remotely about databases???

Well, here’s why:

We have just defined a core element of our member database record, namely the purchase count, and made sure that it works. Now when we read a member record in from our database, we can instantiate it into our class MemberRecord and send it messages to decide what to do. This is OO here, my young padawan, and that’s how we do it.

Yeah, sure, old man, but what about the membership number? That’s the key, according to the story, and you don’t even have it. And isn’t there just a small matter of actually storing and retrieving these babies?

Patience, youngling. Each thing comes in its time. Let’s see what the next tests bring us.

I’m thinking we need something like this: We’ll have two members, say numbers two and six. You are number six. We’ll have them each buy some stuff, in some alternating pattern, and make sure that they each get their freebies at the right time. I’ll start with this test:

    def test_twoAndSixBothGetFreeStuff(self):
        members = MemberCollection()
        for purchaseFromTwo in range(5):
            self.assertEquals(4, members.member(2).charge(4))
        self.assertEquals(0, members.member(2).charge(4))

This isn’t my complete plan, but it is enough to fail. The idea here is that MemberCollection is my database of MemberRecords, and the method member() returns the MemberRecord whose number is given. First cut solution goes like this:

class MemberCollection(object):
    def member(self, memberNumber):
        return MemberRecord()

We don’t really have a collection yet: we just return a new member every time. The result of this, of course, is that we always start at zero purchases, so the sixth purchase is not free.

I can see at least two ways to go from here. I could save a single record and return it always. That would make it work for one person and not for the second, because their purchases would add together. I could make two records, and return the appropriate one, simulating the collection.

There is also a need to add the member number to the MemberRecord, which I will do by extending the constructor. I’ll have to choose between a default number, or modifying the tests. I’ll figure that out when I get there.

Then finally I’ll need a collection, which I can drive out with a third member, or refactor in when the current test works. It’s early days yet. I don’t mind speculating about these things, but I’m not going to do them until the time comes. I think first, I’ll just save one record. That should get this part of the test going, then I’ll extend the test, or write a new one, to break the one-record solution.

class MemberCollection(object):

    def __init__(self):
        self.__record = MemberRecord()

    def member(self, memberNumber):
        return self.__record

That gets our current test working: the single record in the database works for for our one guy. Now I can extend this test, or write a new one. If I write a new one, this one should be renamed. I’ll do that.

    def test_memberTwoStillGetsFree(self):
        members = MemberCollection()
        for purchaseFromTwo in range(5):
            self.assertEquals(4, members.member(2).charge(4))
        self.assertEquals(0, members.member(2).charge(4))

    def test_memberSixDoesNotInterfereWithTwo(self):
        members = MemberCollection()
        for purchaseFromTwo in range(5):
            self.assertEquals(4, members.member(2).charge(4))
        self.assertEquals(4, members.member(6).charge(4), "six interfered")
        self.assertEquals(0, members.member(2).charge(4), "two got no discount")

This fails as expected, “six interfered”. Member six and member two are sharing the same record. I will fix this by making two records:

class MemberCollection(object):

    def __init__(self):
        self.__record2 = MemberRecord()
        self.__record6 = MemberRecord()

    def member(self, memberNumber):
        return self.__record2 if memberNumber == 2 else self.__record6

And our test is green. We now have a database of records. A very small database, mind you, but it is working perfectly for our two members. What’s next? Well, we could refactor a bit, but it seems to me that we’d really be putting in new functionality. So I’ll write another test. I think I’ll process three people at once this time, number two, number six, and number one.

Also, while I’m thinking of it, I think I’ll create some manifest constants for those magic person numbers, as they are not showing up very well in the tests. Here goes:

class NoDBTests(unittest.TestCase):

    personOne = 1
    personTwo = 2
    personSix = 6
    ...
    def test_threeAtOnce(self):
        members = MemberCollection()
        for purchaseNumber in range(5):
            for person in [2, 6, 1]:
                self.assertEquals(3, members.member(person).charge(3), "improper free one")
        self.assertEqual(0, members.member(self.personOne).charge(3))
        self.assertEqual(0, members.member(self.personTwo).charge(3))
        self.assertEqual(0, members.member(self.personSix).charge(3))

Naturally, the self.personOne changes were made elsewhere as appropriate. You can look for those when I provide the final code at the end. Right now, we get the “improper free one” message, as the current implementation of the database will reuse person number six’s record when asked for person number one. (Interesting how that turned out if you were thinking of The Prisoner, but for our database it just won’t do.)

Now we need to build a collection that will work for as many people as we have. One way to do it, certainly, would be to build it up before the tests. We could argue that members will have to already be signed up before their purchases will be counted. Or we could add them dynamically, creating a MemberRecord for any key we don’t already have. I think I’ll go that way, mostly to learn a little more Python. With luck I won’t get in trouble. Let’s see.

I’ll make a collection named members, and a function findMember(memberNumber) that looks for a member. If it finds the member, it’ll return them, otherwise it will create a new one, add it to members, and return it. And I’ll need to add a memberNumber “member” to the MemberRecord. Got that? OK, here goes.

class MemberRecord(object):

    def __init__(self, memberNumber=-1):
        self.__purchaseCount = 0
        self.memberNumber = memberNumber

    def charge(self, amount):
        self.__purchaseCount+=1
        return amount if self.__purchaseCount%6 != 0 else 0

class MemberCollection(object):

    def __init__(self):
        self.members = []

    def member(self, memberNumber):
        memberRecordList = filter(lambda m: m.memberNumber == memberNumber, self.members)
        if len(memberRecordList) > 0:
            return memberRecordList[0]
        else:
            newMember = MemberRecord(memberNumber)
            self.members.append(newMember)
            return newMember

The above code makes the tests green. We now have a MemberRecord that knows its member number, and a MemberCollection that can find MemberRecords by number, and add them on the fly as needed.

You’ll note that I named the finding method member(), not findMember(). Oversight on my part and perhaps we’ll change that now that we are green. You may be wondering how that method works. It goes like this: create a list of all the records numbered “memberNumber”. If the list has more than zero elements (by construction it can only have zero or one), return the zeroth element. If it has zero, create a new one of the desired number, add it, return it.

I’m not unduly fond of this method but it was the first thing I could think of that wasn’t an explicit loop. In retrospect, I may prefer the explicit loop. We’ll see.

You may be wondering about the lambda. That’s an inline function, returning whether x.memberNumber == (the provided) memberNumber. I think that’s how ya do that.

Anyway, now that we are green, let’s review what has happened. Then, time and space permitting, I’ll clean up the code a bit.

Database? Why Yes, In Fact It Is.

Without ever doing any SQL or other SQuirreLy database stuff, we have begun to design and implement an important record in our system, the MemberRecord. We can continue to evolve it, adding meaningful fields and behavior, as long as we want. Because the record is an object, we have at least a decent chance of encapsulating behavior in the right place.

Speaking of that, should our collection be looking at the member number? Or should we have an “isMemberNumber(n)” method on the record? Possibly the latter but the way it is now may lead more more readily toward a database solution. Viz:

We have also done important design on the database “table” that contains MemberRecords. In particular, we have identified that our only needs (so far) are the ability to find a record by its number, and to add a record of a given number. This finding simplifies any actual database we may do in the future to put these records into some persistent collection. If we are the kind of people who use stored procedures, for example, we have just identified the only ones we’ll need.

Again, so far. As we continue to evolve this little app, we may find more. The good news is that they’ll all be nicely located in the MemberRecord and MemberCollection.

And its fast. Our seven tests execute in 0.122 seconds on my ancient tablet PC.

But What About …

Doubtless at this point you have some “what about” questions. I think you can answer them. I’m here to point the way, not drag you along the path. :) But I will take one question just to show that while I am unreasonable, I don’t want to appear that way:

When we finally plug in the real database, how should we do it so that these tests don’t all break?

I’d look into topics like “pluggable behavior”. I could imagine a switch inside MemberCollection that cuts it over to using the real database code, or a different class, maybe OracleMemberCollection with the same behavior but implemented in Oracle. Either way, the switch setting or use of the new class will be plugged into the system at some higher level than we presently have represented.

Please think about other issues, and see how you might address them. For extra credit, try working this way, and solve the issues as they arise. If you keep the objects well encapsulated, I’m sure you’ll do just fine.

Final Code

I think this article is more than long enough already. I’ll post the final code in a separate piece.

P.S. PyCharm is really nice. I am not associated with JetBrains in any way other than as a user. And I like it.

P.P.S. Comments are on, at least for a while.

Thanks for reading!

‘Kate,” Dan said, ‘things are going well. Products are selling, the Marketing people are happy, quality is up, customers are happy. I just have one question.”

“How much of a raise do I want?” Kate said. “It’s nice of you to ask.”

Dan laughed. “Don’t worry about that, there’ll be a nice present in your stocking come Christmas. No, my question is, how can what you’re doing possibly work? As soon as we have a product idea even partly fleshed out, the developers are building it and have a version for us to play with. And then you just grow it from there. How can that possibly work?”

“Do you mean technically? We have tests, refactoring, …”

“No,” Dan said. “That’s your bailiwick. I don’t understand how it can work from a business viewpoint. How can we do all this without nailing down the requirements, estimating everything, and making sure people get done what they said they would do? It seems like you have everyone just winging it.”

“Well, it is a lot like improv,” Kate said. ‘Lots of preparation, practice, refining. Maybe it would help if I told you about my Uncle Guido and his Italian restaurant.”

“You don’t look Italian,” said Dan.

“True enough. I’m in witness protection, but don’t tell anyone. Anyway, my uncle Guido had an Italian restaurant, and he decided he needed a lovely mural on the back wall, to give the place a touch of class. Let me tell you what happened.

“Guido and his pals talked about it a bit, over some of Guido’s famous Questionable Chianti. They decided this was a great idea. They thought about what they needed and decided on a nice Tuscan look, with maybe a grape arbor showing where the Questionable Grapes were grown in the old country, and probably a picture of Great Great Aunt Sophia, sitting at a picnic table or maybe hanging out laundry. They drew pictures of their ideas on bar napkins. Unfortunately most of those were lost, because Questionable Chianti turns out to be a really good bleach.

“Anyway, after they had some ideas, and the headaches were mostly gone, they started to talk with local artists. They learned a lot, most of it bad news. The artists wanted ‘artistic freedom’, or they said that the painting would take a month to do and they needed the restaurant closed for that period. There were all kinds of problems.

“The good news was, this helped Guido understand his needs. He needed to keep the restaurant going, and he didn’t want a big ugly drop cloth hanging there hiding the painting, since there were doors in the wall, to the restrooms and the kitchen. At first he tried to find someone who would paint the whole thing over a weekend, but no one stepped up to that.

“Guido was stumped. He couldn’t have the place looking horrible, and he couldn’t close, but he really wanted his mural. Finally he met one artist who had an idea. ‘What if we let people see how the mural is coming,’ she asked Guido. ‘I’ll work early mornings until you open, and folks can come in every day and see what is happening. They can give you feedback, and you can give me feedback and we’ll build the mural to best suit your ideas and budget.’

“Guido thought a bit and decided that if there were no poisonous fumes, this could be a good idea. He could see the thing taking shape and guide how it went. But he was worried about the budget. ‘How can we keep costs under control,’ he asked.

“The artist suggested this: ‘Let’s set a deadline and total budget. I’ll keep you posted on how much is being spent, and of course we’ll have the picture on the wall to look at. By the time we’re about half-way through, it should be of high enough quality, and have enough picture elements, that we could stop any time. You’ll have more ideas, of course, but by then we’ll both have a sense of how fast we can progress, and you can choose the most valuable things to add or change. You’ll have total control over how the picture winds up, and if you want to, we can stop on or before the money runs out.’

“Guido wasn’t entirely convinced. He wanted to know how he could be sure he wouldn’t be left with a horribly ugly wall. The artist told him that she would guarantee to paint it back over and stop any time he wanted, and said she would start by working in some temporary pigment like chalk, so they could erase and change things easily.

“Guido decided to go ahead.”

Dan thought. “I can almost imagine that working. But it seems awfully chaotic. Did it work?”

Kate said, “Wait and see. The story goes on. Guido was a serious thinking guy, like you are. He had made a lot of notes about the mural and his needs, and many of them were not bleached away by the Questionable Chianti. His first thought was to give all his notes to the artist and have her paint away. I’ve actually seen some of those notes and they were pretty good. Here’s how I remember what he wrote about Great-great Aunt Sophia:”

Sophia was a great beauty in her youth. It was said in our family that Sophia Loren took her name because she looked like our Sophia. Our Sophia had dark dark eyes, jet black hair that flowed down below her shoulders. She had curves to make men cry and she moved like a dancer despite her voluptuous figure.

“Maybe you are Italian after all,” Dan said.

“Silence, I’m telling a story, you dirty old man,” said Kate. “Guido had written more:”

The mural should show Sophia in her later years. She had rounded out a bit, as we Italians will, and somehow she seemed shorter. You could see in her aging face the beautiful girl she had been. And in her eyes you could see the memories of the many men she had conquered. After she married Francesco, she had a long happy married life and many children whom she adored.

The mural should show her as Grandmama Sophia, still energetic and beautiful, still ruling the family gently but firmly. It should let people see the great family heritage that comes down to us and should suggest that just as Sophia created wonderful food for her family, we will create wonderful food for our customers.

The picture should also show that Sophia was frugal, just as our prices are quite fair given the high quality of the food we serve. And she should look hard-working, to show that we will work hard for our customers.

“Wow,” Dan said. “He asks a lot of a picture of a little old lady.”

‘Guido was a poet at heart,” Kate said. “But tell me this: if you were an artist, could you draw a picture of Sophia at age 55 or so, and have it look like Guido’s memory?”

Dan only had to think a moment. “No. I get that she was a lovely lady but I have no idea what she looked like. Hey, wait! This is an allegory isn’t it? Are you talking to me about written specifications?”

Kate laughed. “Caught me. We have the same problem as the artist would have had with Guido’s lovely prose: it doesn’t really tell her what has to be done.”

“What did she do?”

Kate said, “First, she read all of Guido’s pages of notes. Fortunately there weren’t too many, but there were a lot of items that she made notes about:”

The family had a pet mouse named Petra. The mural should have a mouse in it, but it shouldn’t look like the restaurant is infested.

Also, now that I think about it, a sort of Garden of Eden feeling might be good, as our restaurant is a little paradise. And our desserts are very tempting. Maybe there should be a serpent to tempt people. Maybe a Python.

Don’t forget the Questionable Grapes from which we get our famous Chianti.

The family kept goats and sheep. Maybe there could be some hills with animals playing.

I remember one time a wolf or something got my favorite little goat. Should there be a wolf?

Kate took a breath. “Guido’s list went on and on.”

Dan said. “Wait. Guido wanted a python? In a restaurant mural? Of Tuscany?”

Kate said, “It turns out Guido was really hung up on the python. It took a lot for the artist to talk him out of it. And talking is what they did.

“The artist met with Guido after reading all his notes. Guido began by saying that he wanted the artist to write up her design for the mural and her detailed schedule. Fortunately, she had a better idea. ‘Words describing pictures don’t work very well, Guido. I suggest that I’ll draw you some sketches of what I understand. We’ll work with the sketches for a while, and then we’ll start working directly on the wall where your customers can see the mural.’”

Dan said, “Dammit, this is an allegory isn’t it? Your Jesuit education is showing.”

“No, no, this is all true, I swear it,” said Kate, holding up her crossed fingers. “It does relate to how we work on your products though.”

Dan said, “OK, let’s see. You start with whatever written material Marketing has, and then you sit and talk with them. And as soon as you can, you start showing them product pieces, like the artist’s sketches.”

“Yes, that’s right. And, as soon as possible, we get those product pieces in the hands of customers, just like the artist got things on the wall in the restaurant as soon as possible,” said Kate.

Dan said, “Wasn’t  there trouble along the way, like with Aunt Sophia’s looks and the bloody python?”

“The artist handled it pretty well. Her first move on Sophia was to ask for pictures of her at the age Guido had in mind. Naturally, there were lots of pictures in the family albums. She had him pick out his favorites, and over time, she at first sketched some samples, and then later actually did some small renditions in chalk or something. By the time Sophia went on the wall, they had a good idea what she would look like.”

Dan said, “Did they do everything that way, with lots of sketches first?”

“Fortunately not,” Kate said. “One thing that helped was that although some of their meetings had been away from the restaurant, they did have a meeting there pretty soon, because the artist wanted to look at the actual installation. Two important things came out of that.

“First of all, she couldn’t help noticing the two doors in the wall, which Guido had never mentioned to her in conversation or his writing. And she saw that the one led to the restrooms, and the other to the kitchen. She asked whether the mural should incorporate the doors, and if so, whether the one should seem open and inviting, and the other one closed, so people didn’t accidentally go into the kitchen. Guido agreed, of course.

“The artist sketched a couple of ideas … one with the door to the restroom painted to look like an inviting pathway, and one with the door to the kitchen looking like a closed gate. She also showed one with the kitchen doors painted so as to hide the fact that they were doors at all.”

Dan said, “You said two things. What else?”

“The artist noticed that there were tables near the mural wall, not for customers, but where restaurant supplies like plates and silver were kept.  The sketches they had looked at had detail all the way down to the floor, but that detail wouldn’t be visible at all. So being in the real situation had given them some important insights, one of which Guido had thought about but forgotten to mention, and the other one being new to both of them.”

Dan said, “OK, what’s the lesson in this, you Jesuitical minx?”

“Ah, Watson, you know my methods. We try to get together not just with the idea people but with the real customers for our software. In Guido’s case, they were the same person, but by putting Guido in the restaurant with the artist, new understanding came out. When we meet, not just with Marketing, but with our customers and prospects, in the presence of even sketched product ideas, better ideas come out.”

“OK, what happened next to your imaginary uncle Guido?” said Dan.

“Imaginary? Imaginary? For that you have to take me to dinner at his restaurant. Anyway, they worked with sketches for just a bit longer, and then the artist said it was time to start working on the real wall. Guido was pretty scared. He was afraid that if something really ugly went up there, customers would be turned off. The artist agreed, but she was thinking about the Python.

“So she asked Guido if he would prefer to look at sketches and then close the restaurant for a while while she painted the real wall. Right away Guido didn’t want to shut down, but then he realized that no matter how much work she did on the sketches, he wouldn’t have a sense of how the wall was looking until she was finished.”

“Unless he sat there and watched her all the time,” Dan said.

“Yes. And neither of them thought that would be much use, and anyway they would have used up all their time and cost flexibility on sketches. There wouldn’t be much ability to change, just ability to fail or run over.”

“So what did they do?” said Dan.

Kate said, “Just like she had suggested. The artist started coming in in the morning and chalking the mural on the wall. She blocked in big areas, and sometimes drew more detailed bits. I think she did Aunt Sophia in more detail early on, and the Questionable Grape Arbor.

“Guido was worried at first, but the customers loved it. They would ask what was going on, and offer comments. The whole restaurant turned into a kind of little village where people were talking among themselves, and with the staff and Guido about what was going on. A lot of people started coming in more frequently just to see what was happening to the mural.”

Dan said, “Did it all go smoothly into place?”

“Not remotely,” Kate said. “There were lots of changes, some that Guido saw, and some that the customers saw. One time, the artist sketched in the python, and people asked to be seated far away from it. I guess it was a very convincing python. Guido relented on the python at that point.”

“OK, so in our company, you put the product before the customers before it’s finished, and let them give you feedback about it,” said Dan.

“Yes, that’s right. And we build lightly enough that when we get good ideas, we can put them in without too much trouble. That’s where our testing and refactoring play in.”

Dan said, “What about the wall? When she started using real paint, were there still changes?”

“Sure, lots,” Kate said. “Many came from customers, some from Guido, some from the artist herself. In art, you just paint over the bad bits, or scrub them off and start over in some area. In software, we refactor most often, sometimes we replace whole bits. We’d rather refactor, because it’s less expensive, but sometimes we just get it wrong.”

Dan said, “But isn’t that wasteful? If you had planned better, you wouldn’t have all that rework!”

Kate smiled. “Nice try. But the issue isn’t in planning. No number of words would get the right look on Sophia’s face. Guido had to see it on the wall. In fact he had the artist change it several times before she caught his vision of this saintly but still sexy older woman. It kept getting better, and they went back to sketches a few times, but things on the wall look different than in sketches.

“I believe that Guido made some important tradeoffs to get things right. The artist was insistent that they stop on time and on budget. By now, Guido was in the throes of creativity and might have gone on refining Sophia forever. The artist brought him back to earth by reminding him of the schedule. At one point, he actually backed off on some of the animals in the field to get a little more done on Sophia.”

Dan said, “That’s what Susan does, isn’t it? I remember her briefing me one time that she had deferred my favorite feature to get another feature stronger. She had a hard time convincing me but was pretty insistent. In the end, I realized I had given her authority and let her go ahead. Turns out everything went just fine, and she put in my feature in an update later.”

“Exactly,” Kate said. “Building the best possible product in the time and money we have isn’t easy. Sometimes we have to make hard decisions, and sometimes we’ll even make them wrong. No one is perfect. What our approach does is keep everything visible: the product as it grows, the budget as it is spent, the time as it runs out. And we keep the picture on the wall at all times, so when it’s time for the big party, the picture is ready.

“And the mural was ready for Guido’s big party as well. His daughter Amelia decided to get married, rather suddenly I’m told, and Guido threw the biggest party imaginable, at the restaurant. It was just a few days after the original deadline, and of course lots of people came in who had never seen Guido’s place at all. They had a wonderful time and everyone loved the mural. As far as I know, no one asked why there wasn’t a python in the picture.”

Dan thought a moment. “I think I get it. You just work as close to real product as you can, evolving your understanding and the product at the same time. And you make decisions as you go, doing the best you can, and making sure you’re always ready for the next party — I mean ready to ship. Sounds easy.”

Kate smiled. “Pretty close. One thing: it’s not easy. You have to be willing to work in public, with all the good stuff and all the bad stuff right out there to be seen. You have to be willing to make decisions, and to remake them, and you have to work so as to to make change easy. This stuff isn’t really easier than the older ways. It’s just much better.”

“It is much better,” Dan said. “When we first talked, I thought we would have to close the company. Now we have a steady flow of new product and a steady flow of happier customers. Thanks for the story about how you do it, wrapped up in the Mythical Guido’s Restaurant story.”

“Mythical?” Kate said. “Wait until you taste Guido’s veal piccata at the dinner you’re going to buy me. It won’t seem mythical then. And the restrooms? They’re through that door that looks like a hedged pathway into a private part of the garden.”

J Language

J language just isn’t paying off enough for me to invest more time in it. I see no possibility that I’ll be building any real software in J, and the learning curve is a rough one. You pretty much can’t do anything without having a large number of verbs and adverbs at your fingertips.

I regret this, because it is clear that J causes you to think in some interesting ways about your problem and its solution. The J community is interesting as well. I’ve encountered some very helpful people, and it seems that everyone wants to help you get it. This is great. Perhaps one day soon I’ll get to sit down with a J expert and pair on something. If I do, I’ll sure let you know.

iLua System

The iLua system is an interesting little language on the iPad. As my articles have shown, you can actually build little programs that do real things. The language allows you to construct things like objects very readily, using its very interesting internal primitives. It has been fun playing with it.

However, Apple has made it clear, by rejecting certain upgrades from the iLua people, that they will not permit the language ever to get very useful. For example, there was a very simple window/dialog capability put in, and Apple made them take it back out. It appears that Apple intends to enforce their policy that there will be no real software development for the iPad on the iPad.

I’m sad for the iLua people, and I’m sad for myself. But the fact remains that if I’m to build things for the iPad, this is not the way to do it.

What’s Next?

I’m not sure. I’ve had nearly six months off waiting for a new knee and then getting used to it. (Very fine, thanks. I recommend it to anyone who doesn’t mind being felt up in airports.) So I’m looking for things to do that will be fun and, if not lucrative, at least not get in the way of supporting my limited needs.

One thread of that might be some kind of development for mobile platforms, just for the fun of doing it. I attended a very interesting talk at Philly ETE, about using HTML, CSS, and JavaScript to build mobile apps. It looks to me as if this could become a better and more portable way of doing it. So I would like to learn more about that area. If you know of a course relating to it, or want to drop by and pair with us, let me know.

Another thread has to do with coaching and training. Chet and I will still be offering our highly-rated CSM course, and the Agile Skills course. At the same time, these are both aimed at entry-level and near entry-level folks, and I’d like to do a bit of work with more impact that that. Maybe deeper impact is what I mean. I’d like to work with people who have their feet solidly on the ground in Agile, and who want to excel.

So here’s a question to you: what kind of course or workshop might Chet and I offer that would entice you to attend, even if you or your company had to pick up the tab for course and travel?  What could Chet and I bring to the table that few others might? What would the session be like? How long would it go on? What would be really great?

Crazy ideas welcome … thanks!

The Constraints

I’m trying to build a little script in Linden Scripting Language, LSL, the scripting language for Second Life. LSL brings new meaning to the word “rudimentary”.  There is a very limited built-in library of calls, mostly aimed at making things in Second Life move or talk or such. As for web connections, you can send and receive short emails, and you can make an http request. This request has a URL, a few parameters, and a body. Parameters include HTTP_METHOD (GET, POST, PUT, DELETE), HTTP_MIME_TYPE (text/* and apparently application/x-www-form-urlencoded), HTTP_BODY_MAXLENGTH (not supported, locked at 2048), and HTTP_VERIFY_CERT.

The x-www… MIME type will let you URL encode a body with var=value&var2=value2 kinds of strings.

I want to use an existing on-line service, such as Amazon S3 or one of the hosted CouchDB or MongoDB services. Free usage at low volumes is a plus. Low cost usage at low volumes is a must.  Access using this rudimentary capability is, of course, a sine qua non.

I’m just messing around, so do not have the time or inclination to build something on my own site to host it and respond to a language of my own invention. Something out of the box is what I’m looking for.

The Problem

It should go without saying that a scripting language this limited also has limited storage. What I’d like to do is to store a moderate number (around a thousand) strings, by key. The key will be some unique string, the value another string. Keys around 20 or 30 characters, values around 200 or 300.

The script needs to be able to get a value string from the Web system, using the key, to put one back giving the key and value, and to delete a key. Simple hashtable kind of function.

The LSL language has an http_response “event” that will receive any response headers and the response body. My guess is that the content type of the body must be a string. I know for sure that LSL will not receive XML, JSON, Atom, RSS, or PLS content types.

Once the string is back, one can do reasonable parsing of it. Imagine what you could do with simple string functions: pretty much anything if you have the time.

So I’m looking for something on the Web, free or cheap, working with simple get, put, delete by key, storing a fairly small number of fairly short string objects, that can be accessed via a very simple text-focused URL request, returning a text-like response.

But I’m Very Dumb …

It does look to me as if S3 or MongoDB or CouchDB can probably do what I want. But since I have zero tools available in LSL, and little time to play on my PC to try to figure out what the URLs have to look like for get, put, and delete, I’m looking for a system whose docs show what the requests and responses actually look like, or for some examples that do the same.

Very lazy, I know. This is a tiny little hobby, and I can’t be investing hours and hours in it.

Got advice or ideas? Comments are open. Thanks!

“What time’s the meeting?” said Gil. “Is she here?”

“Nine sharp,” said Susan. “I just went by her office. She’s not there.”

“Carl and I have been working in here since 8,” Gil said. “She hasn’t been here for a Diet Coke.”

“This is the day,” Carl said. “She’s late. Let’s go!”

When the team came into her office, Kate was by her window. She spun her wheelchair and scooted to her usual spot at the table.

“How do you do that?” said Gil.

“It’s easy. To spin, push one wheel, pull the other. To go forward, push both. To turn, grab one. To stop, grab both.”

“No, how do you always get to meetings right on time?”

Kate smiled. “That’s easy too: you leave wherever you are as many minutes as it’s going to take to get wherever you’re going.”

Carl said, “You have a Diet Coke.”

“Yes. Yes, I have.”

“It must be warm.”

Kate felt the can. “No, it’s nice and cold. The machine is working just fine.”

“I mean, how did you get it? Did you bring it to work with you?”

“Standard way. Put coins in slot, press the button. I suppose I could have thrown the coins across the room into the slot and tossed my exercise ball against the button, but I’d still have to go get the can.”

“No one could do that,” said Gil.

Kate smiled. “Maybe no one ever tried. What’s this meeting about?”

On his way to the coffee pot, Carl said, “We want to talk about what to do about bugs.”

“Great,” Kate said. “Let’s not have any.” She clapped her hands. “Meeting over?”

Susan laughed. “As Product Owner, I’m good with that. Move to adjourn.”

“Not so fast,” said Gil. “There are always going to be bugs. We have to handle them.”

“I knew it couldn’t be that easy,” said Kate. “What about these defects?”

Gil said, “Well, should we put them into a database? How should we decide which ones to fix?”

Kate said, “Who decides what features you work on?”

Gil said, “Susan, as you well know.”

Kate laughed. “Yes, as we all well know. So … who should decide which broken features to fix?”

Gil said, “Susan?”

“Good idea. Susan.”

Carl, having returned to the table, interrupted. “Oops. I forgot sugar. Sugar, please.”

Kate dipped two fingers into the bowl of sugar packets, and pulled one out. “Don’t move,” she said, and flicked the packet at Carl. It hit him in the chest and dropped into his shirt pocket.

“How did you do that?”

“Easy. Your nerd pack pulls your pocket open.”

Everyone laughed.

Carl frowned. “You never know when you might need a pen. Or a marker. Or a drawing pencil. Or another pen …”

“True that,” Kate said. “So what else about these defects?”

Susan said, “Well, we need to decide just how to handle the scheduling, how to count velocity, and things like that. And if there are a lot of them, how should we keep track of them?”

Kate said, “How do you schedule things now?”

Susan grinned. “Why do you ask questions you know the answers to? I write a short description on a card. If I need help deciding what to write, or need to be sure the idea makes sense, I talk about it with the team. Also, I meet with the testers to work out the correctness checks. We do that about once a week. Then at the planning meeting, I bring enough cards and checks to fill the week, we talk through each one, figure out how many the team can take on, and commit to the work.”

Kate paused, then said,”Why do I ask questions I know the answers to?”

Susan stared. Then, “Oh. If I want a bug fixed, I could write it on a card and bring it to the meeting?”

“Sure could,” said Kate. “But you mentioned something else that you bring.”

“The checks,” said Susan. “The testers and I could bring in some checks for the bug. They always figure out a way to reproduce it anyway.”

Kate smiled. “Sounds good to me. But isn’t there something special about these defect checks?”

Everyone looked confused. Finally, Charlie, a tester, spoke up. “Oh. We should have provided those checks the first time. We screwed up by not having them.”

Kate said, “I’m not into blame, and anyway we can’t go back on our own time line and edit the past. But we do have a good learning opportunity. What can we do to take advantage of that?”

Carl said, “The retrospective. We should look at the defects discovered and see how to do a better job of creating checks for them.”

“Good idea, Carl, and well put,” said Kate. “I noticed that you said ‘we’. Why did you put it that way?”

“Well, we programmers are in those story meetings too, and we help with the examples. Often a bug …”

Kate said, “Defect.”

“… yes, defect. Often a defect comes out in some odd coding case that a programmer might foresee and a tester might not. Anyway, we’re all in this together, so let’s all learn.”

Gil said, “There’s another chance to learn, sooner. That’s as soon as the bug — defect — is found. The testers usually grab a programmer right away, to be sure they understand the situation, and maybe to figure out how to reproduce it.

“Often, we see right away what is going on. That’s the moment to start learning.”

“Yes! Good point, Gil.” Kate looked delighted. “Now you said something about velocity?”

Susan said, “Yes. How should we handle velocity for bu … defect cards.”

Kate said, “Here’s a question I know the answer to: how do you use velocity?”

Susan said, “I use it in my product burn chart, to get a sense of how much we’ll have by the final ship date.”

Carl said, “And we use velocity to estimate how much work we can take on in the Sprint.”

“OK,” said Kate. “Two reasons: know how many features we’ll have by the ship date, and know how much work to take on in the Sprint. Do defects take time to fix? Can we estimate that time for the Sprint?”

Carl grinned. “They do, unfortunately, and we can usually estimate them pretty well. Sometimes we need to take some time to dig in a bit before we know.”

“If you don’t know how long a fix will take, what do you do?”

Carl went on. “Often we just guess how much time it’ll take or how many we can do. That works pretty well. If it’s a really weird one, we usually ask Susan to authorize a time-boxed exploration.”

“A spike,” Kate said.

“Yes. Then we usually know how long the thing will take. In fact, quite often we actually find and fix it inside the spike timebox.”

“OK,” Kate said, “it sounds like you’re totally on top of estimating defects into the Sprint. What about planning toward the release, Susan?”

Susan frowned a bit. “Defects are really a hassle. Usually they’re important, and I have no choice but to fix them. But they don’t fit into my planning at all.”

“Why not?”

“Well, even if I knew how long they take — and I think they probably average out well enough — I don’t know how many there are going to be.”

Kate said, “Couldn’t we just take that average as well? Like if you’re fixing five defects a Sprint, we just assume that if we have ten Sprints left, we’ll have fifty defects to fix?”

Susan said, “Well … we could …”

“So then,” Kate went on, “we could factor the defect fix time right into the planning and tracking. We could bump up the burn chart by fifty, and track the defects just like stories.”

Susan said, “We could, but that makes me feel kind of dirty. Defects are bad. Yes we can learn from them. For example, I learn how to say more clearly how things have to work, and I work closer with Charlie and the rest of the team coming up with examples that help us be sure things work. And I know the programmers are learning how to do their tests better, and better ways of designing things. But …”

“But what?” said Kate.

“I don’t want defects! When I have to schedule a defect fix, even when I can see it’s because of a test that I should have seen, I just don’t get the same thrill I do when I see the gang finish one of my features.”

Gil said, “You just want to punish us, or yourself, for making mistakes. Bugs are inevitable. They just happen.”

Kate said, “Hold on a second. Maybe some defects are inevitable. But aren’t you all learning ways to avoid them and make them less likely?”

Gil said, “Well, yes. Carl and I learned a new way of testing without going to the database last week, based on something Carl read about. That makes it a lot easier to write tests, and much faster to run them. I can identify a bug that I’m sure we would have had two weeks ago, avoided now because we can test better. But still, bugs will happen.”

Kate said, “They will. One reason I like to say ‘defect’, though, is to remind myself that whenever I look back at a ‘bug’ in the system, I can almost always find something concrete that I, or someone on the team, or all of us together, could have done to prevent it. Saying defect reminds me that most of them are up to us to avoid or prevent.”

Gil said, “There will always …”

Kate said, “Yes. There will always be defects. But do this for me. The next time one shows up that was absolutely unpreventable, bring it to a meeting with me. I’ll bet that by the time the meeting is over, we’ll know a way we could have prevented it … and a way that isn’t even all that costly. Meanwhile, let’s pretend that they are all preventable: most of them clearly are.”

Susan said, “So what does this say about planning?”

“Way to keep us on track, Susan. You’re right to feel bad about defects. There are two kinds of demand for work, according to the Lean and Systems Thinking people. Who knows what they are?”

Silence. Then, Susan said, “I don’t know the words, but there are the things our customers actually want and the things I have to do because the software isn’t right.”

Kate clapped. “Exactly right. The words they use are ‘value demand’ and ‘failure demand’. Value demand is the demand for what customers actually want. Failure demand is work we have to do because we are imperfect … because we have room for improvement.

“So I think you’re on the right track, Susan. I would suggest that you should not bump up your target by some expected number of defects, and you should not count defect fixes as features on the burn chart. If you do that, and there are defects, what will happen?”

Susan thought a moment. “My progress line won’t grow so fast. Each defect we fix takes away that much work on a feature. The chart will show us slowing down to the extent that we work on defects.”

“Right!”, said Kate. “Defects do slow us down, and in an unpredictable way. So it makes sense to me not to track them on your chart as if fixing them were a good thing. It’s more of a necessary evil.”

“But wait!” Carl, this time. “Those bugs can be hard to fix. We put in real work on those. We should get credit for that work.”

Kate grinned. “Remember the story lady? She comes in every Friday, looks on your shelves for features, and buys what you have, putting dollars in your pockets and lattes on your desks? What’s that beautiful redhead going to say when she finds a defect fix on the shelf? Am I going to pay for it?”

Carl smiled back. “I guess not. You’re going to say that you already paid for that feature and you want the fix for free.”

Kate said, “That’s right. Making me pay for defects ain’t workin’. I won’t pay money for nothin’. I want my fix for free.”

Everyone groaned.

“Welcome to my earworm. So, do we have a plan? Estimate for the Sprint as usual, don’t put defects on the Product Burn Chart, either in the target or in the progress?”

Everyone agreed.

“Great,” Kate said. “Meeting adjourned. Thanks for stopping by.”

As they started to leave, Carl turned. “One more question, Kate?”

“Sure?”

“Could you really throw your exercise ball and hit the Diet Coke button?”

“Is there money riding on it? Come over to the Bombay some evening and play a game of darts with me. Then we can talk about the exercise ball … and the coin slot.”