Category: Uncategorized

1 min read

Using Personas to Give Backstories to Your Users

August 6, 2014 12:34 pm August 6, 2014

1 min read

Comments Off

1 min read

You already know that roles are crucial to any User Story since they help us put context around who a given feature is for. However, did you know that you can also put context around your roles?

You already know that roles are crucial to any User Story since they help us put context around who a given feature is for. However, did you know that you can also put context around your roles? Using a technique called Personas we can create rich backstories for our users to help us better understand not only their role, but also their goals for using the product.

Creating Personas

In my latest Pluralsight course, Creating Effective User Stories, I cover how to create these Personas using the excellent template created by Roman Pichler. You can see an excerpt from the Personas portion of the course below as well as download Roman's template here.

Personas are an excellent way to better flesh out your team's image of the actual users who will be interacting with the product. Not only do they help us better understand how a user will interact with the features of our product but, more importantly, they help us understand their motivations for doing so.

And, by better understanding both our users' motivations and backgrounds we can better build products that makes sense for them.

Want to learn even more ways to slice your user stories so your team can start working with them immediately? Check out my course, Creating Effective User Stories, for easy techniques that will have you writing better user stories today.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off

2 min read

Uncategorized

Meet the Timestamp Sticky

Jeremy

July 22, 2014 8:26 am July 22, 2014

2 min read

Comments Off

2 min read

It’s no secret that teams who track their progress using physical tools rather than electronic have have a much higher rate of success, but how do you handle of the fear of sticky notes falling off the wall?

It’s no secret that teams who track their progress using physical tools rather than electronic have have a much higher rate of success. This is because low-
fidelity tools such as sticky notes and index cards are much easier to fit into a team’s natural flow, rather than forcing the team into the constraints defined by an
electronic tool. In addition, the more tangible and visible nature of physical items helps the team feel more involved in the flow of the project and, as a result, more vested in the outcome. However, despite all of these advantages, one of the most common push backs to the idea of tracking progress using physical cards is “What happens when they fall off?”.

Despite the fact that this question is so common, the reality is that this concern is largely unfounded. In fact, if teams are working in a reasonable sprint length then the lifetime of your average sticky note will far outlast the lifetime of your average story. But how do you convince people that this is the case?

Meet the Timestamp Sticky. Simply take a regular sticky note, write the current date on it, and stick it somewhere on the wall. Then, whenever someone expresses concern about sticky notes falling off, just point to your timestamp sticky as an example of how long a sticky note can stay on the wall.

This is a great way to alleviate the fear of the sticky note simply falling off of the wall, and help combat the pressure to move to electronic tools.

Want to learn even more ways to slice your user stories so your team can start working with them immediately? Check out my course, Creating Effective User Stories, for easy techniques that will have you writing better user stories today.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off

10 min read

Uncategorized

Stories versus Themes versus Epics

Jeremy

June 24, 2014 2:02 pm June 24, 2014

10 min read

Comments Off

10 min read

When a new team is just beginning to adopt Scrum the difference between stories, themes, and epics always seem to be a source of some confusion. In particular, where stories end and epics begin tends to be a particular sticking point. In this article, we’ll take a closer look at each artifact to get a better understanding of the subtle differences between them so we can better communicate to our teams when each is appropriate for the work at hand.

This article was originally published here by ScrumAlliance.com, in March 2014.

Let’s start with a few definitions…

Stories

A story is a self-contained unit of work agreed upon by the developers and the stakeholders. Stories are the heart of Scrum, and the building blocks of your sprint. An example of a story may be…

“As a salesperson, I’d like to set my password, so I can log into the system.“

Themes

Themes may be thought of as groups of related stories. Often the stories all contribute to a common goal or are related in some obvious way…such as all focusing on a single customer. However, while some stories in a theme may be dependent on one another, they do not need to encapsulate a specific workflow or be delivered together.

Epics

Epics resemble themes in the sense that they are made up of multiple stories. They may also resemble stories in the sense that , at first, many appear to simply be a ‘big story’. As opposed to a theme, however, these stories often comprise a complete workflow for a user.

But there’s an even more important difference between themes and epics. While the stories that comprise an epic may be completed independently, their business value isn’t realized until the entire epic is complete. This means that it rarely makes sense to deliver an epic until all of the underlying stories are complete. In contrast, while stories comprising a theme are related each is still independent enough to be delivered separately and still provide some measurable sense of business value.

Why stories, themes and epics?

When faced with the confusion between stories, themes, and epics many budding agilistas wonder why we wouldn’t simply choose stories in every case. One of the most common reasons is often the sheer size of some stories.

Stories that at first seem daunting often need only to be broken down further into smaller stories that can be estimated individually. To get a handle on how the story can be broken down, try asking your team some of the following questions…

Why does the story seem so difficult to estimate?
Are the individual tasks that make up the story too complex?
Does the story have dependencies on other work that has yet to be completed, or that must be completed by a third party?
Are there technical or business unknowns in the story that are still unclear?

Once you have a feel for where the hesitation originates from you can take another pass at breaking down the story in the context of the questions above.

An example of an epic

Now that we have a feel for ways in which larger stories can be broken down further, let’s take another look at our earlier example of a story to see if it fits this description.

“As a salesperson, I’d like to set my password, so I can log into the system.“

While at first this may seem like a fairly innocuous story, there’s actually a bit to unpack here. For example, a few questions immediately spring to mind…

How does the salesperson first access the system to set their password if their account isn’t fully setup?
Do password strength requirements exist that will need to be enforced?

In order to better tackle some of these unknowns the story above may be better served as an epic comprised of the following stories…

“As an Administrator, I’d like to send an email to a new salesperson containing a tokenized access link, so they may temporarily access the system in order to set their password.”
“As a Salesperson, I’d like to edit my profile, so I may set my password.”
“As an Administrator, I’d like to ensure that all salespeople’s passwords meet corporate strength requirements, so I can harden access to the system.”

There are a few things we may immediately notice about the stories above. First, when broken down each individual story paints a deeper picture of what’s actually happening in the system than the original epic ever could. In the least, this may highlight new interactions with the existing system that may increase or decrease the estimates. For example, in story 1 a developer may respond with “we already have a system in place that allows us to send emails” or “our existing email system only allows for plain-text messages without links in the email.” While neither of these situations may have been obvious prior, either may significantly increase or decrease the estimate.

Next, the stories have an implied workflow to them. A salesperson cannot access the system to update their password until they’ve received their initial email. In addition, the salesperson should not change their password before we can enforce its strength. This workflow has an obvious beginning, an obvious end, as well as a few steps in the middle. For this reason it can often be helpful to think of epics as a Story Arc.

Finally, although the stories comprising an epic may be completed individually, they provide no measurable business value on their own. Shipping the functionality to email new salespeople provides no value if the profile page or the new password functionality doesn’t yet exist. By the same token, allowing the user to set their password without enforcing corporate strength requirements will result in many passwords being invalidated as soon as those requirements are in place.

Creating epics from stories that are just too big

A related rule of thumb is to automatically further break down any story that is estimated above a certain threshold. A good starting point for this threshold may be where your team has historically tended to become inaccurate with their estimates.

For example, imagine that your team’s velocity tends to be relatively predictable when estimating stories of sizes 1, 2, or 3 points but doesn’t have a great track record with stories sized at 5 points and above. If this is the case, then any story estimated at 5 points or above should automatically be broken down further before it's added to a sprint.

In addition, breaking a large story into smaller stories has the added advantage that multiple smaller stories will travel the sprint board faster than a single large story. Though the same amount of work may be occurring in either case, the negative effect on a team’s moral as a single dominating story sets in the ‘In Progress’ column for the majority of the sprint cannot be understated. A vibrant board not only helps a team maintain a feeling of forward momentum, but it also encourages stakeholders to take an interest in checking the board each morning to see the progress the team has made.

An example of a theme

While the example above provides a great demonstration of an epic, themes may still be a bit unclear. Let’s look at an example of a common piece of work that can often be represented as a theme—performance tuning.

Improving the performance of a specific feature of an application often involves making many small improvements towards the end goal of a significant performance improvement. For example, imagine that your application includes a report to display the total sales numbers per salesperson. These numbers may be calculated as of the previous day.

This can be a time consuming report to execute, and as a result the entire application tends to hang for several seconds each time a user runs it. There may be several things we can do to improve this report, which are captured below as stories…

“As a Salesperson, I’d like for the UI of the application to remain responsive while the report is loading, so I may cancel the report if desired.”
“As a Developer, I’d like to pre-aggregate and cache the sales numbers as of the previous day, so they do not need to be recalculated each time the report is ran.”
“As a Developer, I’d like to generate the HTML table containing the sales numbers on the server and send it pre-rendered to the client, so it no longer has to be rendered by JavaScript on the client.”

Each of these stories has the potential to improve the performance of the report. However, while each story contributes to the same overarching goal they do not need to be completed in a specific order. Furthermore, each story could be delivered independently from the others and still provide some measurable benefit to the end user.

For these reasons these stories are best grouped into a theme rather than an epic. Grouping these stories by theme allows us to see the broader goal that each story contributes to without forcing us to approach the stories in a specific order. Also, grouping the stories as a theme also negates the need to hold delivery of the entire group until each story is complete. This give us more flexibility in scheduling and planning as well as in our ability to add and remove stories as the shape of the backlog changes.

In closing

We’ve taken a closer look at each of the Scrum artifacts of stories, themes, and epics as well as dove into the strengths and weaknesses of each. In addition, we've looked at examples of using each type of artifact in practice to better illustrate when it may be the appropriate choice for a situation.

The choices of stories, themes, or epics bring an incredible amount of flexibility to a Scrum team. This flexibility can be realized not only in the creation and grooming of the backlog but also into the planning of each sprint. As a result of having a clear understanding of the differences between each artifact, as well as the flexibility to choose the right vehicle for each situation, teams can dramatically improve the accuracy of their sprint planning sessions. This increase in accuracy can greatly improve the perception of a team’s success in the eyes of stakeholders, which can go a long way to improving their overall odds of success when adopting Scrum.

Want to learn even more ways to slice your user stories so your team can start working with them immediately? Check out my course, Creating Effective User Stories, for easy techniques that will have you writing better user stories today.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off

4 min read

Uncategorized

Using Hit Rate to Gauge Success

Jeremy

April 24, 2014 12:46 pm April 24, 2014

4 min read

Comments Off

4 min read

Velocity usually isn’t the best measure of a team’s progress. Instead, we often find accuracy to be a better indicator of a team’s long term success. Since many teams still find value in having a metric with which they can plot their success, why not use one that measures how accurate they’re becoming?

I've mentioned before that velocity isn't the best measure of a team's progress. Instead, we often find accuracy to be a better indicator of a team's long term success. Since many teams still find value in having a metric with which they can plot their success, why not use one that measures how accurate they're becoming?

Hit Rate is a simple metric that describes a team's accuracy over time. In particular, it measures how close a team's forecasted velocity is coming to its actual velocity each sprint.

We can calculate it using the simple formula below…

$Hit Rate = \frac{V_{Actual}}{V_{Forecast}} \times 100$

Let's Try Some Examples

Imagine that a team forecasted a velocity of 20 points for the first sprint. However, one of the stories turned out to be a bit more difficult than expected which slowed down their progress. In the end, they only completed 17 of the 20 forecasted points. Let's plug that into our Hit Rate formula to see how they did…

$Hit Rate = \frac{17}{20} \times 100 \approx 85\%$

This tells us that the team had a Hit Rate of 85…or that they hit 85% of the velocity they originally forecasted. Note that the $\times 100$ at the end of the formula is only there to give us a nice percentage to work with.

Let's try another one. Imagine that the next sprint the team forecasted a velocity 18 points, but two of the stories turned out to be a simpler than expected. This allowed them to finish early, which let them pull an additional story into the sprint. The end result was that the team completed 2 more points than originally forecasted. Let's plug these numbers into our formula to see how this windfall affected their Hit Rate…

$Hit Rate = \frac{20}{18} \times 100 \approx 111\%$

This time we see the team had Hit Rate of 111%, which means that they actually completed 11% more work than originally forecast.

Hit Rate as a Trend

If either of these cases sound familiar it's because they're quite common in young scrum teams. In fact, if you track Hit Rate from a project's outset you'll likely see a pretty erratic pattern in the beginning. However, if you're willing to pay attention to and learn from your Hit Rate, you should start to see it settle near 100% over time.

What Your Hit Rate Can Teach You

Although some occasional wavering in your Hit Rate is to be expected, sharp trends away from 100% can reveal much about a team's tendencies.

For example, is your Hit Rate consistently trending down? If so, then the team is likely overcommitting to the amount of work they can complete each sprint. Work with them during Sprint Planning to keep the lowest priority stories in the backlog rather than bringing them into the sprint.

What if your Hit Rate is consistently trending up? This likely means that the team can take on more work than they're selecting at the Sprint Planning meetings. Work with them to gradually bring in a bit more work each sprint until the Hit Rate closes in on 100%. This will build their confidence and help them see what they're really capable of.

Using the Hit Rate as a Guide

Even after a team begins to close in on a 100% Hit Rate they still may show some occasional variance. While intermittent variance is no cause for concern, a consistent pattern of moving away from the 100% line in a certain direction should raise your eyebrows. If you're tracking your Hit Rate then this should be easy to spot before it gets away from you. However, if you do find yourself in this situation then just refer to the Actual Velocity from your last few sprints. You can use this number to reset the team back to a more reasonable forecast during your next Sprint Planning. With the Hit Rate as a guide it won't take long to get back on track.

If You Must

Although metrics can be a helpful gauge of a team's progress, we should always remember that they're no replacement for simply getting to know your team and understanding the specific events surrounding each sprint.

However, sometimes it's nice to have a guide that can call out areas where your team can still improve. If you must use a metric, then Hit Rate is the one to look at.

Want to see more about how to make textbook agile work on real teams? Check out my course, Agile in the Real World, for tips and techniques for making agile really work in your organization.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off

3 min read

Uncategorized

Velocity Is Not the Goal

Jeremy

March 26, 2014 7:16 am March 26, 2014

3 min read

Comments Off

3 min read

Teams that post consistently high velocities are often treated as valuable to the organization. However, those teams that tend to be more erratic…posting a high velocity one sprint then falling off dramatically the next…may not be as valuable as they may first appear.

“ve·loc·i·ty (n) – a measure of how many points a team completes during a sprint.”

On velocity

Velocity is often the first scrum metric we encounter. It's a quick way to gauge how many points a team completed during the previous sprint, and how many they'll likely complete in the next. It's clean, simple, and easy to understand. It's also easy for people to get attached to.

On predictability

The reason is that velocity isn't as valuable as predictability. Much of what we do in agile is done with the aim increasing the predictability of our team. The more predictable a team is, the more effectively we can plan into the future. This lets us take risks more intelligently as well as formulate more realistic strategies based on what our teams can likely deliver in a given timeframe.

Teams that post high velocities followed by low velocities are very erratic, and thus can't be planned for. And if a team can't be planned for, then its value to the organization declines.

What's the harm in an erratic velocity?

Erratic velocities can lead to difficult planning for any team. Not only long term release planning, but even short term sprint planning will suffer since the team won't be able to accurately forecast how many points they can complete during a given iteration.

If your team is experiencing erratic velocity, then it may be time to look upstream at your estimating process. Often unpredictable velocities or missed forecasts are the result of poor estimates at sprint planning time. Even exceeding your forecast can be a symptom of over-estimating the amount of work selected for the sprint.

Regardless of whether your estimates are missing up or down, your planning process is suffering.

So what's the goal?

If velocity isn't the goal, then what is?

Regardless of the hype surrounding ‘fast' agile teams, at the end of the day agile is often more concerned with predictability than speed. Posting a high velocity for one sprint isn't helpful if your team can't sustain it. However, posting consistent velocity numbers goes a long way towards great planning.

Does this mean that a team shouldn't strive to produce their velocity? Not at all, it means that improving a team's velocity shouldn't be its goal. Instead, the team should focus on what it takes to improve that velocity. Practices such as continuously improving the engineering process or systematically identifying and chasing out risk will naturally result in higher velocity.

Focus on these things and the velocity will come.

Want to see more about how to make textbook agile work on real teams? Check out my course, Agile in the Real World, for tips and techniques for making agile really work in your organization.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off

9 min read

Uncategorized

Zappos and the Elimination of Managers

Jeremy

January 28, 2014 8:09 am January 28, 2014

9 min read

Comments Off

9 min read

While the tagline Zappos abolishes Managers has gotten great press, in actuality it’s a bit of an over-simplification of what’s happening.

There's been a lot of discussion lately about Zappos' move to eliminate managers from their hierarchy. If you haven't heard yet, go check out the Fast Company article on the transition, which does a nice job of summing up the move.

While the tagline Zappos abolishes Managers has gotten great press, in actuality it's a bit of an over-simplification of what's happening. In actuality, Zappos is transitioning to a new structure which organizes its employees in groups around work to be done, called a holacracy. Rather than following a traditional hierarchical management system, this new structure tends to resemble self-organizing circles focused on individual projects comprised of employees selected for their skill sets.

Are there really no managers?

Perhaps, but that doesn't necessarily equate to no leaders. Reams of business literature have told us time and time again that given any group of sufficient size, leaders naturally emerge. Whether we want them or not, this seems to be a a natural part of human nature…it's in our DNA.

If this is indeed the case, then any attempt to eliminate traditional managers from an organization may be for naught. Left unchecked, it wouldn't be surprising for the exact same organizational structure to reemerge, sans titles of course.

However, this natural tendency to evolve towards structure may not be a bad thing either. Elimination of titles and structure will effectively give Zappos the opportunity to press the reset button on their organization and allow the true leaders to emerge across the company.

Most organizations would kill for this chance.

What's wrong with a traditional hierarchy?

Of course, this begs the question “What's wrong with a traditional hierarchy?” After all, history is full of companies which have become very successful under one. However, the fact that traditional management hierarchies have not killed every company in history doesn't exactly absolve them of guilt, either.

Imagine a small startup, perhaps made up of two co-founders. It's unlikely that one co-founder has asserted themselves as the manager of the other. Rather, its more likely that the two co-founders view one another on equal footing and have come together due to the unique skills each has to offer. In the early stages, even as a few more employees are added here and there, it's likely that this culture will remain largely unchanged.

However, as the company becomes successful and begins to scale this organizational structure will not.

Eventually the leadership of the company will face a day when they need to introduce some sort of management structure to help the company scale. On the whole they will be faced with two options: Flat or Deep.

Flat Hierarchies

Flat hierarchies tend to evolve when individual managers have responsibilities for managing large groups of people.

This particular structure seems to have come into favor in recent years, especially in the tech industry. And for good reason too, as flat structures offer a lot of benefits:

Fewer layers of indirection between the people on the bottom of the organization and those on the top. This tends to lead to more efficient execution of strategies set by the top ranks as the strategy is less likely to be watered down or changed as it passes through each successive rank.
The evolution of cross-functional teams. Since less managers are available in the organization, they tend to accrete more direct reports. One of the common side effects of this is those reports may not only specialize in the same field as the manager, but also those that are complementary. For example, an engineering manager may not only be responsible for managing engineers, but may also be given responsibility for complementary teams such as quality assurance or technical writers. These cross-functional teams tend to delivery more efficiently as the walls of bureaucracy are torn down. In fact, this can even result in reduced in-fighting amongst groups that have to work closely together.
Faster scaling teams. When an individual manager is already responsible for managing 12 people, adding one new employee to their ranks has much less impact than adding an employee to a manager who only has 3 direct reports. This lets flatter organizations scale up teams more rapidly, for better or worse.

However, for all of the benefits of a flat organization they're not without their drawbacks.

Less one-on-one attention from managers. Obviously a manager whose responsible for 12 people has less time to spend with each than a manager whose only responsible for 3. This translates to not only less time for one-on-ones, goal setting sessions, and mentoring, but also less time to simply get to know each employee on a personal level. This can lead to less engagement amongst employees, less attachment to an organization's goals, and ultimately, higher turnover.
Less mentoring to those employees not in the manager's speciality. Let's return to our example of the fateful engineering manager tasked with responsibility for not only software engineers, but also quality assurance engineers, and technical writers. If the manager's background is in software engineering then they are likely to be an excellent resource for their reports who are also aspiring software engineers wanting to take their careers to the next level. However, what about the quality assurance engineers and technical writers? If this is not the manager's area of speciality they'll need to invest extra time in learning how to coach these employees, or risk having an overall lower skilled team.

Deep Hierarchies

Deeper hierarchies tend to evolve when organizations grow towards many layers of middle management, with each manager responsible for only a few direct reports.

This is a bit more of a traditional management structure, and can offset many of the disadvantages of flatter hierarchies described above. Specifically,

More one-on-one attention from managers. More managers means smaller teams which means that managers have more time to focus on each of their people. This can translate into not only more frequent one-on-ones with each employee, but also more time for career planning and goal setting. All of this translates into each manager getting to know each employee better and having more opportunities to imbue the organization's culture and goals to the employee. All of this can lead to more engagement for the employees and, ultimately, lower turnover.
More appropriate mentoring for each employee. More managers also means that it's less likely teams will evolve to be cross-functional. Rather, they'll tend to stay siloed in teams focusing on specific areas of expertise. The benefits of cross-functional teams notwithstanding, increasing the likelihood that a quality assurance engineer will be mentored by a manager who built their career in quality assurance means that engineer can have more relevant mentoring and may reach their career goals quicker.

These benefits aside, however, deep hierarchies are not without their drawbacks.

Less efficient execution of strategies. Just like the classic game of Telephone, no matter how great any strategy starts at the top, the more layers of indirection it crosses through the more likely it is to become weakened or watered down. In the worst scenarios, it may even be corrupted to serve each individual manager's own goals and agenda.
More silos between closely operating teams. If you have more managers the temptation is greater to divide teams amongst specialities. While this can lead to better management of each employee, the effect of these artificial barriers between teams working together each day cannot be understated. Given any team structure the members of those teams will eventually grow to rival and compete against one another. While a little competition is a great thing, it can easily get out of hand and evolve to resentment between teams working closely together. Once started, this in-fighting can be difficult to reign in.
Slower scaling teams. When a multitude of hierarchies exist in a given organization adding more employees can create friction. This is due not only to the greater impact a new employee can have a smaller team, but also to the waves of indirect disruption the change can have on teams that work directly with that team. While this effect may occur simply because the manager is more effectively managing their people due to the smaller team, the effects that it can have on an organization that needs to scale up and down quickly shouldn't be ignored.

Will this scale better than a hierarchy?

Returning to our discussion of a holacracy, one of the biggest questions facing the this structure is how effectively will it scale?

Bearing in mind the points above regarding deep versus flat hierarchies, we know that larger teams, in particular cross-functional teams, tend to scale quicker than smaller and more siloed teams. The holacracy seems to exhibit both of these larger and cross-functional qualities. In addition, we also know that the elimination of a middle management layer could lead to a more effective execution of strategies since each team member has the opportunity to work closer to the top-level decision makers. Whether or not this will be the actual result, however, remains to be seen.

While the lack of middle management may in theory give rank-and-file employees more access to top leaders, it also puts the onus on the employee to take the initiative for their own career growth in the organization. This includes not only mentoring and growth opportunities, but also the adoption and incubation of the organizational culture and values across all employees. Given what we know about Zappos hiring practices and resulting culture this may not be much of an issue for the average Zappos employee. However, not every company has earned the reputation of hiring for passion that Zappos has, how this will effect other, less passionate, organizations remains be seen.

Regardless of the result of Zappos move to a holacracy I have no doubt that we'll have plenty of visibility and insight in the results, due to Tony Hsieh's transparent operating style.

I'm looking forward to seeing what we can learn.

Comments Off

6 min read

Uncategorized

Lessons Learned Using Flyway DB with Distributed Version Control

Jeremy

July 23, 2013 2:52 pm July 23, 2013

6 min read

Comments Off

6 min read

Recently our team introduced Flyway for automating schema changes to our database. Along the way, we learned a few interesting lessons about using a schema migration tool in a team heavily bought into a distributed version control.

Our team uses Git for source control which gives us the ability to easily create branches used only by a few developers. Like most teams, our workflow consists of creating a branch from the mainline branch of code, implementing and testing a new feature inside of that branch, and then merging that branch back into the mainline for shipping.

This can cause a bit of havoc with the default configuration of Flyway, but it can be easily worked around using the tips below.

Prefix your migrations with timestamps rather than integers

By default, most migration frameworks choose to prefix the individual migrations with an integer, as in the example below. When the framework encounters migrations not yet applied to the current database, it starts with the first migration whose prefix isn't present in the database and begins applying them in ascending order.

1__add_customers_table.sql
2__add_email_address_column_to_customers_table.sql
3__add_orders_table_with_reference_to_customer_table.sql

This works great when everyone is on the same branch of code. However, once members of the team begin working on their own branches, the likelihood of a prefix collision increases dramatically.

But, if you choose to prefix your migrations using timestamps rather than integers, then the likelihood of a collision virtually disappears, even across branches. For example, using a pattern such as yyyyMMddHHmmssSSS the migrations above now look like…

20130704144750766__add_customers_table.sql
20130706132142244__add_email_address_column_to_customers_table.sql
20130706151409978__add_orders_table_with_reference_to_customer_table.sql

The timestamp pattern above is precise down to the millisecond. While a highly precise timestamp can lead to hard to read prefixes, the more precise your prefix then the less likely a collision will be.

For best results, you'll want to automate the creation of this timestamp so all members of your team are using a consistent format. To get you started, I've added Gists of both the Ant task and the Groovy task that we use to prefix new migrations.

In addition, note that Flyway also treats timestamp prefixes as integers. This means that if you originally began working with Flyway using integers then you can still switch to timestamps at any point. Since the timestamps are just very large integers, the first timestamp prefixed migration will simply be applied after the last integer prefixed migration.

Enable out of order migrations

Now that we're using prefixes to timestamp our migrations we can rest assured that we won't have any collisions when our branches are merged back into the mainline. However, what about your teammates who stayed on the mainline? Once your changes are merged back in they'll need to apply your migration to their databases but they'll already have higher prefixed migrations.

By default, Flyway will not apply migrations older than the latest migration already applied to the database. That means in our last example your team members will not be able to apply your latest migration.

To allow this, you'll need to enable out of order migrations with Flyway. This is as easy as setting the Out of Order value to true in whichever API you're using to drive Flyway.

Create idempotent migrations

In a perfect world, each migration will only be run once against each database.

In a perfect world, that is.

In actuality, there will be cases where you'll need to re-run migrations against the same database. Often this will be due to a failed migration somewhere along the line, causing you to have to retrace your steps of successful migrations before to get the database back in a working state. When this happens, it's incredibly helpful for the migration to be written in an idempotent manner. If you're not familiar with idempotence, it's simply a very fancy term describing an operation that can be performed multiple times without additional effects.

Let's take the migration below, which adds a column to a table, as an example…

Assuming that your database already contained a table customers, this migration will successfully add the email_address column to it.

However, what would happen if you ran this migration a second time against that same database? Since the email_address column now already exists in the customers table, you would receive and error along the lines of this…

Now imagine troubleshooting this situation if it occurs in a much more complex migration script, particularly if that script is only one of multiple scripts currently being applied to the database.

We can avoid this issue by making our migrations idempotent…or able to be applied multiple times without side effects.

This is done by checking the the state of the database before making any change to it, and only applying the change if necessary. In the example above, changing the script to be idempotent might yield something like this…

This is introduces a fair amount of noise into our script, but at a high level we're simply enclosing our ALTER statement in an IF statement that first checks the database's information schema tables to determine whether that column exists for that table for that database. If the column already exists then the script simply returns, otherwise it is added to the table. This allows us to run the same script multiple times against the same database without issue.

Note that this is actually a bit tricker if you're using MySQL. Since MySQL doesn't allow IF statements outside of stored procedures the statement above actually won't work as a stand alone script. In order to create idempotent migrations in MySQL it's actually necessary to enclose each migration in a stored procedure, execute the stored procedure against the database, and then drop the stored procedure at the end. I've added a Gist here that demonstrates this method for MySQL.

Wrapping up

While I've used Flyway, MySQL, and Git in my examples here, none of this is actually specific to these technologies. In fact, you're likely to encounter the same types of issues when introducing database migrations in any environment using a distributed version control system. Luckily, the techniques above should be supported in any mature migration framework and can save you a lot of potential pain and headaches.

Want to find the perfect next hire for your team? Check out my Agile Developer Assessment on Kand.io, the leading tech-talent assessment platform. This assessment will help you find the perfect fit for your growing agile team.

Comments Off

6 min read

Uncategorized

If You Build It They Will Come, and Other Lies You’ve Been Told

Jeremy

March 13, 2013 9:39 pm March 13, 2013

6 min read

Comments Off

6 min read

We’ve all seen the movies and read the stories, a group of outsiders create something completely under the radar that when unleashed on the world becomes the next runaway success. It’s sexy, it’s exciting, and it’s inspirational to millions of young entrepreneurs. However, there’s one problem: it rarely happens that way.

Too many teams think if they build it the customer will simply appear. But listening to the customer first isn't just smart, it's critical. Imagine the following scenarios…

A group of plucky college kids churn away in their dorm room creating the next website that takes the world by storm. After months of hacking away, the site is unveiled to the world and instantly becomes a runaway success.

A pair of developers spend their evenings around their kitchen table working tirelessly on the next killer app. After weeks of work the app finally emerges from the dim light of the kitchen to become the latest app store hit.

We've all seen the movies and read the stories, a group of outsiders create something completely under the radar that when unleashed on the world becomes the next runaway success. It's sexy, it's exciting, and it's inspirational to millions of young entrepreneurs. However, there's one problem: it rarely happens that way.

Sure there are the exceptions, like the Facebooks and the Angry Birds of the world. But what makes these stories so sensational is not that they're the norm…it's that they're the outliers.

In reality, products that are built inside the vacuum of a dorm room or kitchen often emerge only to a different kind of vacuum: silence. More often than not these products that have been built without customer input tend to miss the mark in meeting, much less exceeding, the market's needs. They rarely provide that “a-ha” moment or “scratch the itch” the customer never knew they had. Instead, these products are met with a lack of interest or impact, if the customer even looks at them at all. The end result is that the product of all of that work in the vacuum becomes yet another casualty of missing the mark.

Getting customers involved

Most companies test their product with customers during beta phases, but this late in development many critical decisions have already been made. These decisions can make it difficult to correct the issues brought up by customers during the beta process. Rather than taking advantage of the opportunity to learn how their product can better serve the customer's needs, many teams insist on spending this time trying to sell the customer on why the decision they made without their involvement is actually better.

Other teams take the opposite of approach. They spend endless hours interviewing customers before a single line of code has been written, instead trying to divine the customer's every want and need. These sessions are incredibly valuable and yield a fountain of user information about the day to day struggles and pain points a customer faces as well as what could make their life better. While this is a great start, unfortunately many of these promising teams then take their new found information away and disappear for the remainder of the development process. By the time they reappear they're often dismayed to learn that the customer's initial vision and the reality of actually interacting with the end product are often nothing more than a study in stark contrasts.

Customer involvement throughout the entire life cycle

What's wrong with these scenarios? In both cases, the team is seeking customer feedback. In both cases the team seems genuinely willing to hear what the customer says. However, in neither case does the team does set themselves up to respond to the customer's feedback in an efficient way.

In the first case, the team can incorporate customer feedback at the beginning but will never have the opportunity to course correct since the customer doesn't see the product again until launch. In the latter case, the the entire product has been developed on assumptions of the customer's needs that were never even validated. The solution is to involve the customer from start to finish. Incorporate the customer in the early visioning of the product, and make the effort to understand their needs. Then, as development begins, continue to seek feedback from the customer throughout the entire process, evolving the product as the customer's needs evolve in light of realizations about what your product can deliver.

If I'd ask people what they wanted, they would have said a faster horse

This quote, often attributed to Henry Ford, is usually cited as an excuse for not asking customers what they want when building a product. But unfortunately the quote focuses on the wrong question to ask.

Asking what a customer wants will yield less than stellar results…instead ask the customer what they need. What one thing would give them the edge against their competition? If money were no object, what would they do for their business? Spend time with the customer learning how they do business. Learn the pain points that have become so routine and such a daily part of their lives and they don't even notice them anymore. And most importantly, encourage them to dream and to challenge you on what they want…not what you want them to have.

But we're disruptive

And I'm very happy for you. However, being disruptive is not an excuse for not talking to customers. Even disruptive ideas benefit from feedback. Don't flatter yourself to believe that your the first person who has ever thought of disrupting the healthcare/financial/education market. You're not. Others have come before you and they have failed. Talk to your customers and understand why those others have failed. There's a reason your customers did not buy from those others, you owe it to yourself…and your product…to understand why.

Take the opportunity

We all want to build great products. We all want to create things our customers want. However, many of us miss an easy opportunity to gain insight into what our customers want by simply involving them in the product development process. Talk to your customers, you'll be glad that you did.

Want to see more about how to make textbook agile work on real teams? Check out my course, Agile in the Real World, for tips and techniques for making agile really work in your organization.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off

5 min read

Uncategorized

How to Build the Right Team

Jeremy

February 14, 2011 8:27 am February 14, 2011

5 min read

Comments Off

5 min read

Did you know that payroll is usually the biggest expenditure of any software company? If you do, then it should come as no surprise that many companies tout the line ‘people are our greatest asset’. However, it can be scary just how little thought those same companies put into hiring the right people for their teams.

A-B-R. Always Be Recruiting

With apologies to Glengarry Glen Ross, the number one rule of the game is Always Be Recruiting. Too many times a team will only think about who they would like for an opening just as that position is posted. This is the easiest way to guarantee that you’ll move faster than you meant to on a hiring decision and inevitably settle for a candidate. And, when you’re working on something as important as growing your team, settling is one thing you do not want to do.

So how do you prevent yourself from being forced to make a decision before you’re ready? By always having a pipeline of strong candidates available at any given time. We build this pipeline by constantly having a recruiting presence in our local market by getting involved in networking events, user groups, or by just encouraging our team members to get out and build their own relationships among friends. We’ll regularly troll user groups looking for people talking about the same types of ideas that we value as a team. Or, we’ll post our open positions to LinkedIn and Twitter which allows us to instantly reach out to former colleagues who may be in the market for a change. The goal is to constantly be building a pipeline of great candidates that is ready to draw on whenever a position opens up.

Know Your Gaps (And How to Fill Them)

At the beginning of each year, we make a list of the skills we have on our team and the skills we need…i.e., our ‘gaps’ for the more business savvy amongst us. The result is a list of skills that we know we need to get better at in the coming year, or that are missing entirely from our team. What’s interesting is that the vast majority of our gaps tend to be very niche skills rather than full-blown positions. For example, rather than ‘DBA' we may list ‘SQL Server performance tuning' as a gap. But unless we think our team of less than 6 can support a ‘Senior SQL Server Performance Tuning Specialist' we're not likely to be hiring it as a full-time position anytime soon. That leaves us with three options:

Grow those skills in our existing team through training and education.
Temporarily bring those skills in from outside through a consulting engagement.
Find a candidate for an existing position who also happens to bring those skills to the table.

Finding a candidate who already has those skills can be a huge win, arguably even more so than growing them in-house since the incoming candidate presumably has real-world experience putting those skills to the test. But how do you find such a candidate? By looking at those who are already in your pipeline.

Imagine that you have an open Software Developer position. Now imagine that you have two equally skilled candidates applying for that same position. Both have several years of experience with the platform that you're currently using and both have logged enough hours in your domain to wield the arcane terminology of your industry like a second language. However, one candidate worked to improve the performance of a large SQL Server backed application at his last job by tracking down multiple bottlenecks between the application code and the database. Remembering our skillset gaps from above the choice becomes clear. By hiring a developer who already has those skills not only do we fill an open position, but we also fill one of our gaps with someone who can share her experience with the rest of the team.

Don’t Hire Yourself

Let me be abundantly clear, in a team-oriented field like software development fit trumps all. How someone's personality fits in with the dynamic of your team, regardless of how amazingly talented they may be, is the single biggest factor as to whether or not they'll be successful. However, just because you're hiring for someone that fits in well with your team, doesn’t mean that you want your very own Mini-Me.

One of the biggest results of the rapid evolution of our field is that there are now tons of moving parts to any software project and countless ways to handle each of them. Odds are that you’ve developed some pretty strong opinions about the best way to do each piece and, odds are, you'll gravitate towards others who share those same opinions. Because, if we’re really honest with ourselves, we tend to believe that our opinions are correct by the virtue of them being our opinions. Hiring someone who shares those opinions can have some great advantages as long as we remember this: those that share our same opinions about techniques and technologies will inevitably share our same blind spots. This is true whether those blind spots are confirmation biases about a given platform, ignorance of emerging alternatives, or just misunderstandings about how an alternative technology works. Don’t perpetuate those same blind spots by hiring them again.

Summary

In a knowledge worker industry there is no single greater advantage that you can give to your team than to infuse them with good people. Why not approach this task with the same rigorous thought and attention to detail that you expect each member of your team to bring to their own tasks?

Comments Off

3 min read

Uncategorized

No deadline. No ship. It’s that simple.

Jeremy

October 23, 2010 7:52 pm October 23, 2010

3 min read

Comments Off

3 min read

If you want to ship your product then you have to set a deadline.

When I was a kid, my big sister and I were addicted to a show called “Out of This World”. The show revolved around a Evie, a young girl with an alien for an estranged father, who could freeze time simply by pressing her index fingers together. I don’t have to tell you that this is the type of concept that takes hold in the mind of an 8 year old and never lets go. I also don’t have to tell you that I spent countless hours in my room pressing my fingers together hoping that ‘maybe this time’ time would freeze for me.

As I’ve gotten older, obviously I’ve realized the fallacy of this idea. The idea that a race of aliens (the Antareuns, I believe) who are capable of freezing time could ever develop interstellar travel is ludicrous.

You see, interstellar travel would be a huge technological achievement, even for the race of extra-terrestrial swingers that existed in the show. An achievement so large, in fact, that it could have only been wrought by the constant pressure of deadlines. The problem, however, is that deadlines and time pressure would be meaningless to a race that could freeze time on a whim. Why march towards a ship date when you can just freeze time until it’s ready? What forces the tradeoff between delivery and functionality when you can simply freeze time until all of the functionality is done? You see, as technologically advanced as the Antareuns were they lacked one crucial piece of knowledge that we learned in software long ago: if you don’t have a deadline, you’ll never ship.

Without the pressure of a hard deadline features will creep, schedules will slip and a released product yours will never be. Part of the reasoning behind this comes back to an idea known as Parkinson’s Law. Parkinson’s Law succinctly states: Work expands so as to fill the time available for its completion. What this means in a nutshell is that regardless of the true time required for a task, however much time you schedule for the task is how long that task will take. Do you have a 1 week feature on the schedule for 2 weeks? Then that feature will take 2 weeks. How about that 30 minute meeting scheduled for 1 hour? Congratulations, you just bought yourself a 1 hour meeting. Like water, any task will invariably expand to fill the time allotted for it.

So what happens when you have a task without a deadline? Then that task has an infinite amount of time allotted for it, and thus, that task will never be completed. If you want to ship your product then you have to set a deadline. No, your product won’t be done at arrival of your deadline. It won’t have all of the bells and whistles you so desperately believe that your customers need and it won’t have that last 0.999% of polish that it would be unthinkable to release without. However, guess what: your competitor’s product won’t be done yet either but it’ll already be in the market. And if their product is all your customers are going to see, then all of the polish in the world won’t save you.

Want to see more about how to make textbook agile work on real teams? Check out my course, Agile in the Real World, for tips and techniques for making agile really work in your organization.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

Comments Off