Photo collage of cell phone, car, and glass of wine

Influencers Part 4: Quality

May 26, 2017  |  Time: 1:00:34  |  Subscribe in iTunes

Quality is difficult to define, so how can you guarantee that your project will result in a quality outcome? There are tools you can use and steps Project Managers can take to ensure quality results. Clearly define quality expectations up front, and make sure all the stakeholders understand and agree. Pay attention to Voice of Customer. Do lots of testing along the way, with a highly critical eye. Ensuring maximum quality goes hand in hand with minimizing potential risk. Three experts on Quality, Jon M. Quigley, Mike Frenette, and Brian Cohn discuss how you can – and should – incorporate “Quality” into all phases of a project.

Listen online or read the full podcast transcript below.


About the Speakers

portrait of Jon M. QuigleyJon M. Quigley

Value Transformation LLC
Principal and Founding Member

Jon M. Quigley PMP CTFL is a principal and founding member of Value Transformation LLC, a product development training and cost improvement organization established in 2009 (www.valuetransform.com). Jon has multiple master level degrees from City University of Seattle, as well as an engineering degree from University of North Carolina at Charlotte. He has been employed by PACCAR and Volvo, and is an experienced tier one supplier to the automotive industry (Ford, General Motors, Chrysler and Harley Davidson and more). Jon has won awards such as the Volvo 3P Technical Award and the Volvo Technology Award, and has secured 7 United States patents. Jon has co-authored or contributed to more than 12 blocks. The titles range in topics all associated with project management and product development including quality tools. He is on the MPM advisory board of Western Carolina and Forsyth Technical Community College. He is available for speaking and guest lecturing on project management and product development and can be reached at Jon.Quigley@valuetransform.com.

portrait of Mike FrenetteMike Frenette

Corvo Project Management
Owner

Mike Frenette, PMP, CMC, I.S.P., SMC has been engaged in information technology and project management for more years than he cares to admit. He has been employed by utility, oil and gas, and manufacturing organizations, and several consulting firms over the last few decades. Mike currently owns Corvo Project Management, a PPM consulting, mentoring and training company. He has a penchant for staying up to date with methods and processes that help organizations achieve their strategic goals, and enjoys sharing what he has learned with anyone who will listen. Mike is a frequent speaker and trainer on project management topics locally, nationally and internationally at in-person conferences and virtual sessions. He has proven to be a serial volunteer through roles on boards and committees, contributing to the inner workings of the Canadian Information Processing, the Canadian Management Consultants Atlantic, PMI Nova Scotia and PMI Global.

portrait of Brian CohnBrian Cohn

Danfoss Power Solutions
Process Excellence Manager

Brian Cohn is Process Excellence Manager at Danfoss Power Solutions focusing on the development of new products and services. He has held design engineering, application engineering, project management, and product management roles, at firms in the manufacturing, aerospace, and defense segments for over 30 years. Most of his experiences have been in business to business environments and he is passionate about improving both the effectiveness and efficiency of new product development.


Full Podcast Transcript

0:00:02 Kendall Lott: Greetings, PMs. I'm talking to you from the top of Bear Mountain, overlooking the Hudson River, north of New York City. You might ask why. I've engaged on an Appalachian Trail through-hike, started last April. Now, it's day 26. I've been planning this project for over a year and I have to say, my planning has served me well, mostly. I found out that I've had to refine some of my resource requirements, and I have to watch my own energy levels, and of course, the weather is always a risk. I've had to slog through some cold, muddy and rainy days, some very windy days, misty days, and now some blazing hot days. Did I mention the bugs? Still all in all, it's an adventure of a lifetime and I'm learning a lot, and applying some project management practices. I'm actually doing some forward and backward passes as I plan multiple days resupply.

0:00:50 KL: If you want to follow my progress, I invite you to check out my Facebook page at www.facebook.com/flipfloppintheat, that's F-L-I-P F-L-O-P-P-I-N T-H-E A-T. Meanwhile, I will continue to post PM Point of View episodes from the trail. They may not show up as regular monthly episodes, but we do have a few in the pipeline, so stay tuned. If you notice the sound of the birds or the wind whipping across the microphone, you'll know why, I'm out in the woods. Just keep on listening.

[music]

0:01:23 Speaker 2: You cannot assume the product is good, you find a way to understand the real nature of the product. And for me that's maybe take a negative attitude. There's a break in it. There's a failure in there, I just need to find it.

0:01:33 Speaker 3: People don't understand that quality is not testing. It is fixing things so that when you do test it, they work.

0:01:40 Speaker 4: We have a very strong quality system within our organization and we have a fairly simple definition. It is conformance to requirements.

0:01:51 KL: We often think of quality as one of those intangibles, like, "You know it when you see it," but you can't quite put your finger on it. You can't measure it…or can you? If you really want high quality deliverables, we need to find a way to bring quality into all of the many areas of the PMBOK, including the core areas of cost and schedule.

[music]

0:02:09 Speaker 5: From the Washington DC Chapter of the Project Management Institute, this is PM Point of View. The podcast that looks at project management from all the angles. Here's your host, Kendall Lott.

0:02:20 KL: The influencers I've gathered for this episode have all devoted some serious time and thought to the subject of quality, which is intimately tied to risk. Maximizing quality will help to minimize risk. There are tools and methods we can use to ensure quality in every phase of our project. You'll hear about things like Voice of Customer and FMEA, which stands for Failure Mode and Effects Analysis; a deliverable expectations document, the DED; and of course they talk about testing, lots of testing, along with probing critiques at every step of the way. With a background in marketing as well as engineering, 

Jon M. Quigley is a principal of a product development, training, and cost improvement organization called Value Transformation LLC. For more than two decades, he's worked in product development, primarily in the automotive industry and specifically on embedded software. Jon has won technical awards for his products and secured several US and international patents. He has also co-authored focus books for software test professionals.

0:03:19 KL: Can you give us a little explanation about what embedded software is? 

0:03:22 Jon M. Quigley: Your phone, that's an embedded product. There's a little micro-controller, microcomputer inside there, it's embedded inside the product and gives the product its ability to do the things that it does. The electronic control units on your vehicle, your brakes, they're controlled by something that's called probably a brake controller, micro-controller that monitors the brake. Your instrument clusters... Got a micro-controller in it, which reads information off the data bus, and which is a communication method from between all the controllers on the vehicle, and takes action accordingly. Your microwave probably has a micro-controller in it; if it's high-end, your refrigerator might have a microcontroller in it. Many come with control features that are software in the product but it's not like a laptop or a desktop, it's not that kind of software. It does something in real time.

0:04:11 KL: So, is this the stuff that might be getting hacked in the future in the internet of things? 

0:04:15 JQ: One of the things, yeah.

0:04:17 KL: So it's just moved front and center for us? 

0:04:19 JQ: That would be one of the attributes to a good quality system right? You would not want somebody to be able to log in, get into your car, either Wi-Fi, or through a communications port located on the vehicle usually used for diagnostics, and do some kind of tampering with your vehicle.

[music]

0:04:41 KL: This is about risk. This is quality tied to risk because the sense of, you put this out in a bazillion of... That's probably a term of art, bazillion, but a lot of microwaves could be out there, a lot of car brake instrument clusters are out there...

0:04:56 JQ: By the time you find out you have a problem, it's after you've already produced a large volume, if you haven't done some things up front to understand the nature of the product.

0:05:04 KL: So this is software and volume, and not because it's a platform that you can reach out like maybe a large IT company could reach out and either push an update or issue a new package.

0:05:17 JQ: Yeah. Virtually or something like that. Yeah, exactly. Here's what makes it a little more difficult for a project manager. You pretty much know the schedule and you kind of have an idea whether you're late. You also have a pretty good idea about monitoring the cost of the project. The thing with quality, unless you take a very active role like you do in those others, you're not going to know until you have launched it. For example, if I don't do any testing, I can assume that the product is okay. If I don't look at the product with a critical eye, then I might not have found anything. But the reason I haven't found anything is because I haven't looked hard enough.

0:05:52 KL: And it matters here again to me is because the risk is so much more highlighted. It's software embedded in hardware in volume that you can't recover. So once it goes out, it goes out.

0:06:00 JQ: Yes, a lot of times... Yeah. A lot of times in automotives, for example, they're not connected to a Wi-Fi generally speaking where you can just update on the fly, but when you start having material involved, shipping game DVDs and material like you put in a car and recalls and volumes, that can be big money and a lot of risk, a lot of people can get hurt.

[music]

0:06:29 KL: The PMBOK has quality management. We can talk about quality management. You seem to reference that there's a sense that if people are not as active in it because they can't monitor it as well. The idea that if you have a failed product and you have re-worked it that's costly, is generally known. The idea now that in embedded software, for example, it is a much higher risk, so why is this any different? 

0:06:50 JQ: So there's always this pressure of getting the thing out the door as quick as you can, this is part of that reward calculation and the project managers are... Companies are focused on bottom line and meeting those kind of objectives and sometimes you can, at least from experience, you can see that the quality, it's maybe not monitored sufficiently to make those judgement calls. It's a hyper focus on the cost and the delivery date and if you don't really know how to go around critiquing the product or you have too many people who are optimistic critiquing the product, then you may walk away with insufficient understanding of the product quality for those reasons.

0:07:32 KL: Is it the people don't know to do it or they don't do it well or what you just said, the people who tend to do it have a positive bias? 

0:07:37 JQ: It's all three of those things.

[music]

0:07:44 JQ: They deliver an automotive-grade product from Voice of Customer, all the way through to the manufacturing floor and out to the customer without making a mistake. There's millions of interactions there, that's thousands of lines of requirements, documents and maybe even lines of code. On top of that, there's interpretation issues which is not necessarily an error in the documentation but it's a misunderstanding kind of thing. And it's possible some of those things are just downright wrong that we wrote, so we built something like it's written but it will never work, it's not going to do what we expected it to do.

0:08:21 KL: Maybe a misunderstanding of what testing is for. Testing is to show that.

0:08:23 JQ: For me that's exactly it. Testing is a significant component. My experience is a lot of people think testing is the last thing that happens and that's not true. Testing should be integrated all along with your development, and every time you do an iteration on your way to the final product, you should have somebody critique it. And that person that critiques it should have this mindset, not that this is a good product which is what the developers are going to tell you, because they think they make no mistakes. I'm sure they gave it their best effort, but this person has to have the idea of, at least the tester, that there's a bug in there. I know it's broken, I just need to find it.

0:09:02 KL: So this is what you mean by getting active in the way that you said schedule and cost are monitored actively and early and we have a good handle on that? 

0:09:09 JQ: Right. Keep doing that and those iterations that you build. I was a software guy once. I wrote code and you write a little bit and you test that little bit, and you write a little bit and you test that. That's the way it works best from my experience. There's a lot of evidence to support that too, there's a lot of studies that say don't sit down and write the whole thing at one time and hand it to a tester because it's a mess. Product development is filled with things that you don't know about what the end product's going to be and building those iterations and playing with those iterations and by playing I mean testing, that's one form of playing with them, you learn something about the product and how it can be used or possibly be used and the risks that are associated.

[music]

0:09:55 KL: How do you see this play out in automotive? 

0:09:58 JQ: Well, testing is significant component of automotive quality but there's things that happen before that. Design reviews of the specifications or the intent, there is reviews of the way we plan on going about achieving the goal of the Voice of Customer, that's called Quality Function Deployment, we can use some other matrix tools to help with that, then there's tools that critique the design beyond the paper requirements document or even that include that, for example a specification review or something like that, then we have tools called Design Failure Mode Effects Analysis which is a thought experiment on the product before you even have the product.

0:10:42 KL: Let's talk about some of these because I think these are really interesting. Talk about the Quality Function Deployment.

0:10:47 JQ: That's a tool that you use to compare... You're looking at the quality of the product, how you believe that it will be worked and try to evoke any interferences between the quality that can be delivered and the feature itself, the quality expectation and the feature. I've used up one little... My preference is the Pugh Matrix to do those kinds of analysis.

0:11:12 KL: Tell us about the Pugh Matrix.

0:11:14 JQ: It's a weighted management decision matrix where you break out the things that matter and you compare the design against those things with some kind of numeric value. You can compare multiple designs from multiple approaches to a particular part of the design and you look at them from a variety of perspectives, cost could be one, ease of implementation could be one, repeatable quality level or complexity could be another kind of thing, some attributes in those categories that you would look and you start weeding out things that won't work or that have too much risk associated with them and you start coming up with ideas that more closely align with your calculation of risk-reward.

[music]

0:12:00 KL: Quality is really a function of risk, it's that connection of scope to risk almost, to me, because what you're really describing was, is it too risky to do? So you set up the features you want, you weight them according to what you care about most, and then you run all your designs against it. So you were really doing a validation or selection process of design, viewing it as a quality problem or quality opportunity while eliminating risk. We're not going to build the thing we don't want. So I guess maybe people are going to take on the risk that they do want in those cases.

0:12:32 JQ: Right. And in order to take on or understand that risk-reward calculation, you have to have done some analysis of what the risk could be. The reward part is pretty easy, we estimate that it takes $2 million to do this project because we've done this kind of thing before. It takes 18 months to do because we've done something similar to this before. We might be off a little bit but it's going to be kind of close. So we know what's in that bucket, we probably now have a general idea of what the market will bear, from either historical record or studies on the market itself. Then you have to start thinking about what can go wrong in the product. And for me, that starts early on with the concept selection. You pick the wrong concept and you open up a whole set of risks.

[music]

0:13:21 KL: I'm going to go back to one other tool. One of the fun ones, it's failure mode effect analysis. FMEA. This is an old technique. What part of the quality process do you look at this? 

0:13:31 JQ: Two parts. Look at the design intent and you have engineers that have been around for a while that've seen things. And you think about, "Okay, what can go wrong? What is the function and how can it fail?" And if it fails, how severe would it be? How probable would it be? And how often will it happen? 

0:13:48 KL: It's like risk analysis but at the process level or at the function level.

0:13:52 JQ: Well, in this case it's at the design. We call that a design Failure Mode Effects Analysis. So you're critiquing the design. You're right, there's also a process version of this called the process failure mode effects analysis and that's reserved for the manufacturing and the material handling parts of the product. But in the case of the design, you're looking at a specific part of the design and say, "Okay, here's the door, it's designed to keep the water out. What if water gets in? What would make it possible for water to get in here?" And then you'll start looking at, well this could fail and that could fail. And you'd look at how severe that would be and how probable that would be. Multiply severity probability in occurrence numbers and you get something called a risk priority number and that tells you whether you should do something else about this or maybe it's not severe that you can ignore it.

0:14:37 KL: Kind of those classic risk analyses at that point. Speak to something earlier, you said the people doing this need not to be the ones who are optimistic, they are intentionally looking for how can the following design statement not work. How can it break.

0:14:51 JQ: Yes.

[music]

0:14:58 KL: One of the other thoughts that come out in other Project Manager's Point of View here sometimes is the role of the project inside the enterprise, how it fits in a portfolio in a program. What is project management in terms of leadership, how do you talk to executives about it and things like that. So projects sit down at the point of creating the mechanisms of change, creating the mechanisms of producing products. And when you hit brand risk, I'm suddenly thinking executives would probably pay attention to this, that they're being told a product needs to be good or that a project manager is going to save some money by cutting back on some of the overhead, as it were, related to quality or add to it. They're just kind of, "Tell me the price of this thing and get it done and get it done on time." Have you seen this within enterprises that project managers are able to elevate the conversation? 

0:15:42 JQ: You have to come with evidence, you can't just say, "This is what can go wrong." So you have to spend time to actually do some exploration, and you can maybe theorize what could go wrong but my experience suggests that people need to see an actual wrong event before they'll start taking those things maybe a little more seriously, with a little more strength behind it to match the cost and the schedule stresses.

0:16:09 KL: How do you show the data for that? 

0:16:11 JQ: Well, yeah, that's the thing. You have to have the right mindset to start. [laughter] You have to have the thinking that quality is a significant part of our culture, quality is a significant part of our end result. I'm going to focus on the cost, the schedule AND the quality. I'm going to work out whatever measurements in quality that I need to pay attention to, to improve my chances of success.

0:16:34 KL: We have to break the tyranny of hyper management of cost and schedule. When you hit brand as an executive I start thinking like, "Wait hold it, that's hard to undo."

0:16:42 JQ: Once you start going down that road, we as people start thinking your quality's not so good then... It's like for example, let's look at Samsung, their latest phone. I liked Samsung before that. I'm not sure I'd buy one based on what I've seen the latest failure is for a little bit.

[music]

0:17:03 KL: When you have a project that is being developed and being executed in a global team, that's obviously adding complexity, I think because of different cultures as well as different time zones as well as more requirements for handoffs and communication. I would imagine some sort of standards have to be started to put in place.

0:17:22 JQ: It's funny. You took the conversation here because I was just thinking that we haven't talked about that and that's a significant part. It is, it's not just the documentation, it's not the, "I don't speak English as my primary language and you don't speak Hindi." There are logistics challenges with managing the communications between the sites. If I'm building a product and the development is distributed the same, I'm building a system... You're doing some of it, I'm doing some of it, somebody in Sweden is doing some of it and somebody in Germany is doing some other piece. The theory is all these pieces are going to have to work together at some point. That's our intent, it's a system that's supposed to work together. If I make some decisions on the product and you make some decisions on the product based on your interpretation of the spec and somebody in Germany does the same thing when we start putting the parts together, the thing might not work. And finding the root of those problems takes time. And time is something that's often not accounted for in the schedule because this is not part of the work, it's part of, "Okay, what happened here?" This is more akin to problem solving that might not be in the work or schedule.

0:18:31 KL: It's kind of overhead or transactional time.

0:18:34 JQ: Yeah, it's transactional time, exactly. My experience is, no matter what kind of distance tools you use to keep the team linked, they help and you can still get good results out of that but it's not as simple as distributed across the globe and don't pay attention to it.

[music]

0:18:54 KL: So is the question that managing quality becomes harder or just more costly? 

0:18:58 JQ: I think it becomes harder. Every one of those interactions are now, you have almost no informal communication to avert things. There's almost no chance of serendipitous water cooler conversation that, "Oh wait, you're going to do that with your part? Then I need to make this change over here." There's a lot more coordination effort required, configuration management now becomes a really big deal. It's another set of actions you have to take.

[music]

0:19:29 KL: All of this seems to be consistent with PMBOK but somehow there's something different about this. There's something you need to focus on more about this from an engineering perspective. Can you talk at all about how this is beyond or different or a subset of the PMBOK discussion? 

0:19:47 JQ: Yeah, it's a subset. When you see things like perform quality control, that's...

0:19:55 KL: How can you say no to that? Of course we should do that.

0:19:57 JQ: Yeah, exactly. That's part of the quality part of the PMBOK. But there's a lot of things underneath that that are required and some of those will be engineering-centric, the same with quality assurance. Your project plan probably has some kind of auditing requirements or should have some auditing hooks in the milestones into the schedule and you're testing aspect should be included in the schedule as well and those things kind of help you understand the product quality as you're moving through.

[music]

0:20:32 KL: So as you're talking about this from an automotive perspective and the Voice of Customer, so you're coming out of the whole area of automotive engineering here. How can you link those together for us? You're part of the Society for Automotive Engineers. Is that the proper name? 

0:20:45 JQ: Yes, that's it. It's a Society of Automotive Engineers, yeah.

0:20:48 KL: Do you see that there's shared information between the two kinds of project management and its institutes and the automotive engineers and their societies? 

0:20:56 JQ: There should be, because if either one of these things failed then you probably have a failed product. If the engineering's poor and you haven't had some significant amount of rigor in the engineering aspects, not just the design but the critique of design all the way to manufacturing, then you're going to fail. But if I have a project that is not doing a very good job of coordinating the activities and finding the right resources and talent and giving them the right time and tools and money they need to achieve that objective, then I'm going to fail also. I have an MBA in Marketing and a Master of Science in Project Management and an Engineering degree. I have all of those because I understood that any one of those things goes wrong, chances are good the whole thing goes wrong. You design a great product but you can't deliver it to the market, that's a fail. You can't deliver it cost effectively to market because your project management's not good, that's a fail.

[music]

0:21:53 KL: Yeah, I'm hearing some of the engineering ideas coming into the project management here. The idea of Voice of Customer and how we're going to get to design right to begin with and being able to link the production processes or the design processes using the project management approach.

0:22:12 JQ: It's a very systematic approach to all of this and metric driven. You don't assume the product is good. You find a way to understand the real nature of the product and for me that's maybe take a negative attitude. There's a break in there, there's a failure in there. I just need to find it. And if you don't find it, either you need to keep looking or perhaps maybe you've got something of good quality here.

0:22:31 KL: Well if only we had a module on this under the PMBOK, how they have a specific area on a risk manual or something like that.

0:22:37 JQ: If you're a project manager, you could probably do yourself some good if you were to understand the type of metrics for engineering and put those things in place for your project, especially when it comes to quality. Usually the schedule is done for you. You have a Gantt chart that's being tracked and you certainly have the finance people giving you feedback on your budget. Then you find somebody who can help you with the quality part.

[music]

0:23:05 KL: So remember folks, when it comes to ensuring quality, the power of negative thinking is a good thing. Look for John's latest book, co-authored with Roopa Shenoy, Project Management for Automotive Engineers: A Field Guide. It ties together some of the elements he discussed in this segment, starting with Voice of Customer all the way to launch and post launch. The book is available through SAE, Society of Automotive Engineers, as well as at wwww.valuetransform.com. Also, you can contact Jon through LinkedIn.

[music]

0:23:47 KL: Based in Halifax, Nova Scotia, Mike Frenette is the owner of Corvo Project Management which delivers project management in agile training. Before starting his own company, Mike worked at CRS Systems and before that he was with Anderson Consulting. Before that Michelin Tires. In short, he's been around the block. He calls himself a serial PMI volunteer and is a frequent speaker on PM and agile topics presenting over 50 times in the last decade. He recently wrote a guide on Quality for one of his clients.

0:24:18 KL: When you're writing this guide, what's kind of the thematic thing that you see the most as you're trying to consult to others about how quality should work? 

0:24:25 Mike Frenette: Well, I think there are a couple of areas of focus especially since I was in Information Technology. But in Information Technology, when people talk about quality, they're usually talking about quality control. They're very seldom talking about quality assurance. There's a huge focus on testing and that, you and I both know, is quality control. In other words, you don't build it right, you find out you built it wrong and then you go away and fix it and there's a famous quote from Harold Dodge, "You cannot inspect quality into a product" which over the years has been shortened to, "You can't inspect quality in," which I think is a good maximum for anybody in Information Technology and probably in any other industry that's producing products.

[music]

0:25:22 KL: How do you define your quality assurance approach then? 

0:25:25 MF: In information technology, we try to define what level of quality is required before we ever start creating anything and really, the only way we can do that is to engage our stakeholders directly and to break down as we're supposed to do on any project, what we're doing into the various components. People of course normally use a work breakdown structure to do that and to define the deliverables that the project will produce. So once we know what we have to produce, we shouldn't just run away and start trying to produce it without finding out from the people who will be reviewing the product when we're finished producing it, what they're expecting. So a couple of different ways to do that, if you're in a waterfall methodology and even if you were using an agile framework, you could create some sort of expectation document.

0:26:26 MF: And then the guide I wrote, I called it a deliverable expectation document in which the person who is going to review the deliverable and sign it off helps define the level of quality required for the product. As we all know, quality does not equal perfect. So it's important to understand where that needle is on the gauge from zero to 100 in terms of the deliverable that's being produced, in fact, in terms of every deliverable that's being produced, and it can't be done in vague terms. You can't sit down with a client and say, "Okay, where do you want the quality level? 80%? 70%?" Not gonna work, so it means going through it. It might mean creating models, what the product might look like or the deliverable might look like. Getting agreement and if that means bringing stakeholders in some other parts of the organization to do that, then that's what you do.

[music]

0:27:30 KL: When you talk about setting expectations of qualities, I'm hearing the kind of stakeholder management issue. I mean in the end, the client has to agree to it and sign off. How do you balance the subjective from the objective? In other words, it seems to me that when I look at quality and how it's defined and discussed in the PMBOK when you're looking at thresholds, it kind of even a six sigma approach, there's some sense that there's data and you can measure, is this thing big enough, round enough, does it provide functionality, whatever, versus the client doesn't like it. What their expectation is, is there a balance there or does this boil down to the client either likes it or they don't and it doesn’t really matter if there's an objective measure of quality? 

0:28:12 MF: Well, that's why you have a Deliverable Expectation Document. The client is helping you create that and in your mind, as the producer of the deliverable, you have to think of your client reviewing it with that Deliverable Expectation Document, which they have helped to create and that they have signed off on and they are identified as the signatory of the final product so they should have this deliverable expectation document. Let's call it a DED in front of them when they're reviewing the final product. If you can see that it contains some guideline for the producers of the deliverable and for the client when they go to sign off, something that is clear, then you know you've got a good document on your hands. If you have something that's fuzzy that doesn't contain any measurements, that doesn't contain anything you can go by and you're afraid that when you produce the document, even though the client has signed off on the DED, they may not sign off on the deliverable then, you know got a problem and you have to do something to fix that.

0:29:15 KL: In terms of in the planning stage, you're saying you have to fix that on the front-end of it.

[music]

0:29:23 MF: Then we get into the realm of requirements management, requirements of solicitation, making sure the requirements you produce do in fact address quality, contain visual models that will help you and your user understand what has been recorded as the requirement and what they're agreeing to. A user, a client, a sponsor, a stakeholder, they're not normally going to talk about quality, they're going to talk about expectations. They're going to talk about the product. They will tell you lots of things about the product, some of them will be very specific. They don't have quality in the back of their mind. Now they do when they see the product.

[chuckle]

0:30:10 MF: Because if it doesn't work, they know it. But you need to address it up front and say, "Hey, you know we're producing something here that, as we speak to you now, you can't even visualize." “You can't see it. You can't touch it and when we produce it, maybe you can see it but you can't hold it in your hand.”

0:30:25 KL: And to me, that's where the rub is because it's not really about whether it works or not. It may work, particularly because of... You said software. It may very well work, but they may not like how it works.

[music]

0:30:42 MF: The DED is a little different from the requirements documents mainly because it focuses on a single deliverable and it may have a few extra items in it that will help you and your client understand when the product has actually been delivered.

0:31:00 KL: What kind of elements are those? 

0:31:01 MF: Well, it totally depends on the document. You might be talking about clarity. You may even be talking about things like the size of a document, if it's a document you're producing. You might be talking about user experience if it's a piece of software. You may be talking about so many different things and when you look at a deliverable and you're thinking, "I have to produce this," or, "My team has to produce this," those things that you have to define are going to come to mind. Otherwise, you can't really know what you're going to produce, can you? You must be able to come up with metrics for what you're producing or you're sort of headed for failure I would think.

0:31:46 KL: It strikes me that quality seems to be the very first, the way we're talking about it here, be the very first thing the client will be reacting to upon seeing the, or the stakeholder, upon seeing the product, the output of a project, right? It's like their first thing they're going to feel, right? "Oh, I don't like it." "I like it." That's speaking to what you're describing as their sense of the quality.

0:32:07 MF: Yes, and we tend to think of quality in Information Technology as the absence of bugs.

[chuckle]

0:32:15 MF: And a client, probably the first thing they think about when they see an Information Technology product is, "Does it look nice? Is it user friendly? Do I know what to do without opening a 500 page instruction booklet? Is it pretty?"

0:32:32 KL: "Is it cool?" is what they want to know and then, "What can I do with it?"

0:32:37 MF: And then they try to use it. If they have trouble using it then that's one thing. That's lack of quality in the user experience. If they can use it but it doesn't do what they think it's supposed to do, then there may have been a lack of quality in production of the requirements. If it breaks, then obviously there's something wrong with it and that's what we try to have not happen through all this test planning, test cases, scripts and so on.

[music]

0:33:11 KL: What that caught me thinking about was the very first thing they react to, of all the things project managers plan for, is quality. But I don't think quality is the first thing people plan towards, generally. You know, I come in and I'm thinking charter. I'm thinking laying out the highest level, little bits of everything I have to handle, in the charter, and I start thinking scope, right? It's requirements and it's work breakdown structure. What is actually being required of this thing and what do I have to do to get there? And then we start working, resourcing and scheduling and all that. Or how do you see this integrating at the beginning of planning? 

0:33:43 MF: Well, I think if you look at each one of the knowledge areas, quality is something that has to come into play for every knowledge area. And of course in planning, we're creating that project management plan which is how we're going to treat every knowledge area on the project. When we think of that, we must have quality in mind. If we don't, then we're not going to plan properly for quality.

[music]

0:34:12 MF: So we can have a quality management plan as part of the project management plan, but we have to have a little mini quality management plan, if you will, in every one of the other knowledge areas.

0:34:26 KL: How's that work? 

0:34:27 MF: Well, for example, if you're talking about scope, we know that in a project charter, scope is going to be fairly high level. At some point in time you're going to break it down using a work breakdown structure and defining deliverables. And at the point where you are there, you have the deliverable in mind, then you define it in more detail, especially if you're using an agile methodology, you may not know much about it until you get to the point where you're about to produce it. So how do you define that? You see to me, that should be in the scope management plan. How will we ensure that we have quality in scope? And it's not just the work breakdown structure, it goes beyond that. It goes to the DED's that I mentioned or in an agile project, you might say, "Well, it's not a DED, it's something else." Maybe it's a Definition of Done. Maybe it's acceptance criteria. But at some point in time you get really detailed about that.

0:35:29 MF: If you look at time, well, quality processes take time. That's why they cost money. Would you want to have quality processes that chew up 90% of your project budget? Probably not. Would you like to have quality processes that cause the quality assurance part to cause a lot of products being sent back for rework? Probably not. Because it all costs time. So how will quality work into the time part of the project management plan? Same thing with cost, very similar to time of course, it's another resource that we consume on projects. How will we deal with it when rework has to be done? How will we deal with a client who says, "What are all these strange quality control processes that you have? Why do you have those? That costs me a lot of money. I didn't ask for those. I asked for deliverables to be produced. What are all these quality processes that I see all over the plan." How will you answer that? 

0:36:36 KL: How do you answer that? And what's the justification? 

0:36:40 MF: Yeah. Well, if it's 95% of the project budget, there's no justification.

[music]

0:36:49 KL: From an actual practitioner's perspective, how do people minimize the cost to maximize the actual production of a quality product or output? 

0:37:00 MF: I always go back to the PMBOK on this because you have quality planning, you have quality control, you have quality assurance. So quality planning is all about what processes we're going to use for quality control, what processes are we going to use for quality assurance. If something fails testing at quality assurance time, how will it be sent back? Who's going to pay for that? We have quality control processes. Why would product be not produced well enough, why would it have to be sent back? And we know that there will be product that is not produced well enough and will have to be sent back. So if we talk about that upfront, and talk about quality control processes to reduce the incidence of that happening, then I think that's where the conversation starts. If it starts with, why do you have all these quality processes here and look at it, it's 20% of the project budget, then you're going down a road that's going to be pretty tough, I think. If you talk about the importance of quality in a product, if you talk about the definition of quality with your sponsor, with your stakeholders and engaging them in defining what a quality product means, I don't think anybody's going to believe that they can do that at zero cost.

[music]

0:38:21 KL: I'm wondering if that's an area where there might be benchmarks or some guides for that? 

0:38:26 MF: I'm sure there must be a cost of quality in various industries. If you are talking about the aerospace industry, I would expect the cost of quality would be a lot higher than if you're talking about accounting software development.

0:38:41 KL: Even the question of the trade-off of the cost of recovering from a defect versus the cost of avoiding it to begin with, and a willingness to take that on probably has to do with the nature of the product.

[music]

0:38:58 KL: This just strikes me really talking about a method of looking at risk management.

0:39:02 MF: Well, quality is certainly separate from risk. But producing a quality product is really important and not producing a quality product is a big risk. So how do we mitigate it? I think DEDs, definition of done, acceptance criteria, engaging the user in the definition of quality and definition of what the deliverable should look like when it's finished, all those things are ways to reduce risk. And when you make up your risk identification list, if you have quality in the back of your mind, you should get a long list of risks that maybe you might not have gotten if you weren't considering quality.

[music]

0:39:47 MF: How do you make sure you've got quality requirements, so that once you start digging down into what has to be delivered and trying to get quality expectations from your client, how do you get that into requirements? The document that PMI created, "The Business Analysts Practice Guide." I recommend to everybody, it's chock-full of models, methods, processes for describing requirements. And I think we're heading toward blueprints, making requirements visual. I really think if we make requirements visual, then we'll have a much higher probability of successful project.

0:40:34 KL: And why is that important? 

0:40:36 MF: Well, it's the old, "A picture is worth a thousand words." So if you want to describe something, if you can put it in a picture or a model or a spreadsheet or a drawing, something people can look at, they can't touch and feel it, but at least they can look at. So one example is your usual wireframes, someone says they want a system to do x, y, and z. So you have to create a wireframe, well make it look like something that they can relate to. It makes it much easier for the client to understand it. Data, we've had data models around for ages, swim lanes workflow, another type of model that can be used to really help clients understand whether you understand their requirement. Plus it also helps clients understand whether they have expressed themselves correctly.

0:41:34 MF: One of the projects I was on dealt with a utility, but we brought up swim lane diagrams, we brought up data models, and anytime anybody said anything about some regulatory requirement, it was very often related to quality. So at that point, you write on the diagram and you stick an arrow on it pointed at something relevant and say, "You have to have this number of those" or "You can't exceed this number of those" or "You better not have any of these or you're in deep trouble." You can understand how you would build systems to have some big red flags when some of those thresholds were crossed.

0:42:20 MF: People don't understand that quality is not testing. It absolutely is not testing. It is fixing things that when you do test it, they work. Writing your code, following standards, doing everything you can to make sure that this thing is going to work at the time it's tested, but you have to engage with the users. So if I had to say two things about quality, it's that. Engage with the user, define what you need to do to produce a quality product right the first time as they used to say at Michelin Tires, and they have a wonderful quality program there, and then work toward producing it right the first time based on what the client told you they needed, what their quality expectation was, and then worry about testing and keep your fingers crossed that there's no rework because that's your goal, zero rework.

0:43:19 KL: My guess is this is the hidden risk mitigation cost...

0:43:23 MF: Yes.

0:43:24 KL: That people don't see or don't want to see.

0:43:27 MF: Yes. You could be right there. Everybody knows they expect a quality product. Not everybody wants to pay for it and so there's a fine line there. But you're producing something. If it doesn't work properly, you've got a big problem. I would recommend that anytime anybody's writing a project management plan with their team, that they have quality top of mind in every one of the knowledge areas. I think you can apply quality in each one of them. You can understand the cost of quality and you can understand the impact of poor quality in every one of those knowledge areas.

0:44:06 KL: So remember, engage with the users so you can make it right the first time. To ensure that all of the stakeholders are on the same page, draft a Deliverable Expectation Document. Use models and graphs or anything that will help everybody better imagine the desired outcome. And keep quality in mind at every step of the way. Look for Mike's articles and webinars at projectmanagement.com. There's a lot of interesting stuff there.

[music]

0:44:38 KL: Brian Cohn has been involved in product development for more than 30 years in engineering, project management, product management, and application engineering. Currently a process excellence manager at Danfoss Power Solutions in Minnesota, Brian works with project teams to ensure that they do a great job in developing new products. He also works on a number of continuous improvement initiatives in that realm. One topic he has particular passion for: Quality in product development.

0:45:05 KL: How do you define quality? 

0:45:07 Brian Cone: We have very strong quality system within our organization. And we have a fairly simple definition. It is conformance to requirements.

0:45:16 KL: Is there an issue about market alignment there then? So the requirements then, is that what the market wants or what the executive wants or…? 

0:45:24 BC: We have a process that we go through at the beginning of each project to uncover the requirements from the market. Of course we then make some decisions about what our products will do. It’s that we may have many customers that have conflicting requirements or need different levels of performance in a particular parameter. We have a process that we go through to decide what level of performance we're going to provide because every feature that we add to a product adds cost, it adds risk, it adds failure modes so we really try hard to find that product that meets the sweet spot of the market.

[music]

0:46:15 KL: So talk to me about how your role in the oversight of the product development practices touches in quality and what does that look like in the way you end up rolling it out in that role.

0:46:25 BC: In our business, we support people that do large construction and agriculture equipment, have a mindset very similar to the automotive industry, so they have expectations that the products from us that they put in their machines will work perfectly when they're in and work perfectly for many many years.

0:46:44 KL: So there's a certain sense of consistency over time that matters here. This is not just a project quality in the sense of just getting through the project, but that the product itself has this, I guess it's designed quality, for being used for many years.

0:47:00 BC: Yes. So I think there are certainly aspects of project quality that we need to work on. There's a quality aspect in terms of ensuring that the products that we develop provide the features and the functionality that the customers expect. We always think about quality as conformance to requirements and really even beyond that, conformance to expectations. And so we have a lot of things we do to gather the Voice of the Customer, understand their expectations, turn those into requirements, and then ensure that the products meet those requirements. Generally, one of the elements of those requirements is durability.

0:47:47 KL: Some of our other speakers on quality have all hit on this one thing that is not in my mind specific or explicit in the PMBOK for example, which is this Voice of the Customer.

0:47:55 BC: Yes. Very important part and we have a lot of things that go into safety. I mean, if one of our products doesn't work correctly and we have, for example, a wheel-loader that goes downhill on its own when it's supposed to be stationary, it could kill somebody. It's really important that we build that quality to ensure that we meet the safety functionalities of the machines that we go into.

0:48:26 KL: But take me farther. Does this tie, in your world then, with a product lifecycle to how it's released to the market and, you know, promotion? 

0:48:35 BC: Absolutely. When we develop a product, it is at least a three work stream process where we do, of course, the engineering work, the assembly work, and the operations, and of course everything that we need to do to ensure the quality of our incoming materials from our suppliers. But equally important, and many people would say even more important, is the launch of the product. Our products typically get sold to dozens, if not hundreds, of different customers with their different applications. So there's also a lot of work that has to be done to understand the needs of not just a single application of the product but many applications of the product.

[music]

0:49:27 KL: Can you talk to us about how you've seen products go bad with respect to having missed a quality step? 

0:49:34 BC: What we see, and it's not actually that uncommon and it's not necessarily a quality step, but a customer will use a product, and maybe even something that's been used successfully in the market for many years, in a new application that works the product harder or a different duty cycle, something like that, and they may experience unanticipated failures in the field. Our customers are very often fairly large concerns that have extensive field testing of their machines before they take them into the market. And so it's not uncommon for us to have a product on a machine for up to a year of field testing before they start production so that we can uncover as many of those unanticipated events before they go to market and before they expose themselves to significant risk.

0:50:36 KL: What's the trigger for release in a case like that? How do you walk them across the, "You won't be exposed to risk now?"

0:50:42 BC: From their perspective, it is field trials. So they'll have half a dozen or a dozen machines that they'll put in the field. We have very extensive test plans and test expectations again. We work very hard to take a look at the requirements and fully trace those into tests that we do. We have a lot of products that have 10,000 hour life expectations and we have ways to accelerate that, but in our product development, it's not unusual to have tests that take three to six months to do... To torture test our products before we even let them into the hands of the customer. It's very important for us to get the requirements right the first time so that they can go in and they have a very stable product that they put in their machine.

[music]

0:51:40 BC: Of course, almost all projects are investments of some sort. And if I'm an operations manager, I'm used to thinking of risks in a different way. I'm thinking of low probability, big impact scenarios. My key supplier has a fire or flood. In fact we had a supplier who had a fire and then three months later had a flood at their factory. On the other hand, if I take the sale's side, I've got a set of risks of customer A may not buy a whole lot, but customer B may... But I also have a lot of statistics that I can have a fair amount of variability in any one customer, but in aggregate, it's fairly low risk.

0:52:25 BC: On the other hand, in a project we generally have a fairly large number of, maybe not huge impact risks, but they're there. And they tend to be fairly high probability. And so if I look over the course of this project, and I've got 18 months, which is our typical project duration, is I might have a project plan that says if everything goes well it's going to take me 12 or 15 months, but I know I need to put in three months or whatever the right amount is for the project, a buffer for those events which, I cannot tell you which one is going to happen, but I do know that some of them will. And it's really hard for management to get around that idea of needing a project buffer, to have a high probability of when we're going to release. Because it's very important to the organization, and especially to the sale's side of the organization, to have some confidence of when this product is going to be ready to sell.

0:53:29 BC: There's a lot of work that the project managers need to do to explain to the organization who isn't used to project risk, that they need to put that buffer in. And then there's the struggle of managers, "No, take that out because we need to get it done as quickly as possible and we don't want you planning your project to take 18 months when there's a possibility it can be done in 15." But then somebody locks in on that 15-month timeframe and then everybody gets disappointed when it takes the 18 month that we really expected it to take because we knew that some of these risks events are going to happen.

[music]

0:54:13 KL: So is this really about communication? It sounds like there's some analysis that gives you the number but the next key is to communicate this.

0:54:19 BC: It's a matter of finding a way and to communicate it that the organization can understand. I find it interesting because if I turn this around and I think about what we do in operations, any one product in our lines typically takes somewhere between 15 minutes and an hour of assembly line time to build and yet somebody places an order, and we promise we're going to ship it to them in six weeks or maybe eight weeks, sometimes it's even 12 weeks to do that, and they seem to accept that and to me that... There's a lot of buffer here, and it's sort of bit understood as the normal way of operation and it's just not accepted, not understood in a product development perspective and it's something that we need to find better ways to communicate to the organization that this is the right way to plan projects.

0:55:13 KL: We've had a speaker on this, actually a guest on the podcast looking at this, kind of looking at it from a lean perspective, the idea of stripping buffer out and essentially there's a lot of schedule buffer you immediately put in when you do this and essentially scope buffer, when you do things in an agile perspective, trying to figure out what you're having to adjust in scope and try to find a way to, as a portfolio of projects, being able to use resources and enable teams in a more effective way.

[music]

0:55:43 KL: By the way, the podcast I referenced there was Advances Part Two: Beyond the PMBOK, check it out.

[music]

0:55:51 KL: I don't know that people think about quality as soon as the words 'innovation' come out of someone's mouth. How would you incite or entice our project managers to think about innovation and quality at the same time, or should they? 

0:56:03 BC: They need to. It’s that if you go to this very simplistic definition of quality, which is conformance to requirements and in it by analogy conformance to expectations and ability to do the job that needs to be done, we always start by identifying what is the job to be done by the product and then finding a way that does that in a unique way, in a way that provides some additional benefit to the customer. But it has to do it in a way that meets their expectations and meets their needs. So it has to be designed in and thought in from the get-go.

[music]

0:56:49 KL: Where can people find out more information in your mind around how quality and product management, project management all come together? 

0:56:58 BC: One place to start is the Product Development and Management Association, similar to the PMBOK. They have their own guide for product development and it guides you to many things that would be done in the realm of product development, to ensure that you end up with a product that meets requirements. There's also the Society for Concurrent Product Development, SCPD, that people could go to that brings some of these things to mind.

0:57:28 KL: That would be very helpful. I think, for people who want to follow up and hear how they can take their project management lens and maybe go farther into this space.

0:57:35 BC: To me, the key message is that first, the standard project management skills and toolbox are critical to success in product development. We've seen many instances where we've pulled an engineer out and said, "Go manage this project" and it doesn't go very well. Looking at risk from many lenses is really critical to success and looking at risk from strategic risk, is this project appropriate for the company to... What are the things that are the risk to the schedule, the risk to fulfilling the business case down to really detailed risks that come up through the FMEA process and to make sure that this is a team sport, a team process to do it. It is not engineering doing something and throwing it over the wall to manufacturing, throwing it over the wall to marketing to sell it. We all have to do it together and I think that's something that can make being a manager of product development projects a little more challenging is you have these people with very different mindsets that you have to bring together and turn into a team to develop something that is a success for the company.

0:58:48 KL: So they have the right skills to lean on there for this? 

0:58:51 BC: Absolutely.

[music]

0:58:54 KL: I encourage you to check out some of those resources Brian just mentioned and look for his webinars on projectmanagement.com.

[music]

0:59:03 KL: Meeting expectations means getting quality expectations into requirements. It means keeping the voice of customer in your headlights, from the starting gate all the way through the final output and launch. To deliver a quality product, you must include a quality management plan in all the knowledge areas of your project plan, beyond scope time and budget but to include risk, communication, staffing, and acquisition. 

Special thanks to my guests Jon M. Quigley, Michael Frenette, and Brian Cohn.

0:59:33 S5: Our theme music was composed by Molly Flannery, used with permission. Additional original music by Gary Fieldman, Rich Greenblatt, Lionel Lyles, and Hiroaki Honshuku. Post production performed at M Powered Strategies and technical and web support provided by Potomac Management Resources.

0:59:49 KL: PMPs who have listened to this complete podcast may submit a PDU claim, one PDU, in the Talent Triangle Technical Project Management with the Project Management Institute CCRS system. Use provider code C046, the Washington DC chapter and the title PMPOV 0039 Influencers Part IV, Quality. Visit our facebook page, PM Point of View to comment and to listen to more episodes. There you will also find links to the transcripts of all of our episodes. You can also leave a comment at pmiwdc.org/contact and of course you may contact me directly on LinkedIn. I'm your host, Kendall Lott, and until next time keep it in scope and get it done.

1:00:29 S5: This podcast is a Final Milestone production distributed by PMIWDC.


About the 'Project Management Point of View' Podcast Series

© PMIWDC and Kendall Lott

This podcast series is a collection of brief and informative conversations between MPS President, Kendall Lott, and a wide variety of practitioners and executives. His guests discuss their unique perspectives on project management, its uses, its challenges, its changes, and its future.