Agile methodology has radically transformed the process of software development. It has a number of advantages over the waterfall model that explains why it’s proved to be so compelling. What many people do not realize is that the agile methodology isn’t just something applicable to coding projects alone.

In fact, agile is simply an iterative project management technique that can be applied whenever an activity can be clearly identified, defined, prioritized, broken down into releases and each release further disaggregated into smaller tasks known as sprints. Therefore, agile data governance is possible. 

Drawbacks of the Conventional Approach 

Unfortunately, the agile method is not used as much as it could for establishing a data governance program. Many organizations follow a linear path that begins with creating a data dictionary, defining data domains across the enterprise and appointing a data steward to run the entire process.

This approach however piles up enormous work on the data governor, data governance council, data steward and stakeholders up front. Yet, this approach doesn’t deliver any meaningful or commensurate value in the short term. With this asymmetry of input versus output, it’s not hard to see how the initiative can quickly run out of steam before it has taken off the ground. 

Why Agile Data Governance 

With agile data governance, the organization identifies small, bite-sized initiatives that hold strategic importance for the business, put these through multiple iterations to perfection then build on the successes to embark on progressively more initiatives. That way, the data governance council can keep all stakeholders abreast of decisions, milestones and wins while keeping work inputs focused and limited.

The business benefit is realized much faster and thus generates even more data governance enthusiasm among stakeholders. That said, one of the dangers of agile data governance is that due to the small scale of focus initiatives, there’s a risk that the decision-making process will be inadvertently confined in a silo. However, this can be overcome if the entire program is infused with a ‘big picture’ mindset from the onset.

A Tightly or Loosely Structured Framework? 

Data governance frameworks can be close-knit or loosely structured. And both types of frameworks can work. The appropriateness of the different approaches to data governance depends on the culture, maturity, objectives, structure, size, complexity and technology of the organization in question.

To get it right from the get-go, the foundational phase of a data governance initiative must include: 

Discovery 

Discovery in order to identify opportunities, quantify business value, identify stakeholders, assess sentiment, prioritize focus areas, develop goals and define a data governance model roadmap. 

Foundational Implementation 

Foundational implementation that defines strategic objectives of data governance, educates stakeholders, obtains senior management buy-in and designates data stewards.

Once these initial steps are complete, the organization’s data governance council can make an informed decision as to whether to go with a highly or loosely structured framework. The highly structured team would be supported by dedicated roles as well as clearly-defined tools, templates and process.

A less structured approach would have a loosely knit team of individuals who systematically work the roadmap to accomplish the data governance goals. 

Two Ways to Apply Agile Data Governance 

Irrespective of the approach to data governance an organization chooses to take, the agile technique may be applied. There are two ways agile data governance would be useful. 

Data Governance Framework 

The establishment of the overarching enterprise-wide data governance best practices framework. This is the rock upon which your entire data governance program rests upon. It identifies a data governor (i.e. the primary CEO-mandated data governance champion), the data governance council (headed by the data governor) and a steering committee.

The framework also establishes the strategic objectives, the data governance charters, policies, procedures and training. All these elements of a framework can be broken down into stories or modules that are subjected to agile iteration then prioritized, completed and rolled out over a fixed period of time. 

Actionable Data Governance 

Actionable governance refers to the activities and artifacts that must be defined, planned and eventually executed within the distinct projects of a data governance program. For instance, if you want to introduce a new data source to a process, actionable governance in this context will require that a data set’s stakeholders be known as well as what ways the stakeholders use the data. 

Examples of Agile Data Governance Activities 

To give you a more practical feel of what agile data governance looks like, here are some of the key activities that would be involved. 

Engage Stakeholders 

Start by including data owners within the program’s steering committee. Data owners would ordinarily be representatives of business units who are charged with ensuring the accuracy of data.

Next, ensure these data owners as well as data stewards and users receive training that raises their understanding of the need for governance, how the business will benefit and what specific role they are expected to play in the entire process. This equips participants with the knowledge needed, communicates expectations and delivers a consistent message. 

Capture the Agile Work 

By referring to the organization’s data governance roadmap, establish what the ingredients of the data governance framework are and determine how these can be captured as agile stories. You can proceed to formally write down and detail the stories in an actionable format to ease execution.

For instance, you could go with “As a marketing manager, I want near real-time market information and customer behavior data so that we can target products to the right audience.” 

Make It Visible 

With the agile work stories captured and defined, it’s now time to make them visible. You could do that via a simple non-tech technique such as Post-It notes, an Intranet dashboard, a large TV screen, an agile project management software or any combination of these.

The idea is to make sure everyone understands what needs to be done and can readily refer back to the work plan if they want clarification. These stories should be accessible to the steering committee so they can conveniently keep tabs on what is being tackled at any point in time. 

A Data Governor to Prioritize the Work 

The data governor is the person mandated by the CEO to drive the data governance program organization-wide. They chair the data governance council and, often, the steering committee as well. Even though they may have another role to play within the business (such as COO or CIO), they have a robust understanding of governance which allows them to effectively champion governance initiatives.

Thanks to the backing from the organization’s CEO and board, the data governor is best placed to determine the priority of work. The highest priority work will be the one that has the greatest impact on the business’ strategic goals. 

Implement Minimal Viable Product (MVP) 

What is the least amount of work that can be seen through to the end in order to establish data governance within the business? This is the essence of agile data governance. The answer to this question will vary from organization to organization. It will primarily depend on what triggered the data governance initiative in the first place (such as a caution from an external auditor or censure from a regulator) and what work is considered high priority.

To formally define the MVP, the data governance council should classify work stories into ‘must have’, ‘should have’ and ‘nice to have’. One or more of the ‘must have’ work stories should be the core building blocks of the MVP. 

Continuous Delivery 

Don’t boil the ocean. Even for small and medium-sized businesses, it’s not possible for one to effectively implement a data governance program in one fell swoop. And that’s irrespective of whether you use the agile method or not.

However, the inherently staggered nature of data governance implementation lends itself to the agile technique. You could for instance plan to complete and approve work in some format on a fortnightly or monthly basis. This could run iteratively until the work’s outcome conforms to the governance framework’s expectations. 

Continuous Retrospection and Improvement 

Even when work is deemed complete and signed off by the data governance council and steering committee, that doesn’t signify the end of learning. There’s always room for improvement. Ergo, once work is complete, team members must regularly come together to identify areas of improvement. This retrospection must be integrated in the framework, so it’s triggered automatically. 

Conclusion 

The agile methodology may have transformed the software development process over the last two decades, but it’s still not well understood within the IT industry let alone among non-techies. Driving a data governance program with the agile technique is going to take plenty of training, practice, engaging external experts and learning from early failure.

The good thing is the payoff for agile data governance is substantial. Data users will be much happier to participate in the program and will develop a strong conviction that indeed the strategic end goals of the data governance best practices framework are achievable.

As the data governance program matures and focus starts to shift to strengthening the pillars of an agile data governance framework such as data management, data security and data privacy, then new standards, policies, procedures and controls will be required as well as updates to existing ones.

This is the final post in a five-part blog series on data governance. Make sure you check out the rest of the series to learn more about:

1. What is Data Governance?
2. Data Governance vs Data Management What’s the Difference?
3. Data Governance best practices, policies and rules.
4. Data Governance Framework: What Is It and Why You Need One.
5. Agile Data Governance? Yes, It's Possible.