How AI Will Change Project Management: From Knowledge Support to Organisational Learning

APM Project Management Training

The episode explores the transformative potential of AI in project management through a three-tiered framework: knowledge assistance, decision

Key takeaways

• AI's first tier, 'knowledge assistance,' acts as a specialized encyclopedia but requires context-rich prompting to prevent hallucinations.
• The Model Context Protocol (MCP) serves as a 'USB drive for AI,' allowing models like Claude to plug into tools like Notion or Procore.

Main topics

• Three layers of AI implementation
• Model Context Protocol (MCP)

Notable quotes

The MCP is like your software version of a USB drive.

Conclusion

AI offers a path from simple search to intelligent project agents, pro

Transcript preview

Speaker 1 (0:00) Welcome to another Parallel Project podcast. My name is Paul Neba and with me is James Gardner. Hello, James. Hi, Paul. How are you? Yeah, really good today. So we're going to talk about AI and how AI can be useful for project managers today and in the future. But before we get into that, you're a bit of an expert in AI. Do you want to tell us who you are and what you do and your podcast channel and other content that you produce? Speaker 2 (0:27) Yeah, so I mean, you say I'm an expert. I certainly wasn't an expert. I wouldn't really count myself as an expert now, just someone who's insane curious about this. My background is as a QS, actually. So I'm a chartered QS. I'm a fellow of the Royal Institution of Chartered Surveyors. And I spent most of my career as a QS, delivering projects, mainly in the education sector, throughout most of my sort of 20-year careers on the tools. working for many universities, lots of Oxford colleges, Imperial College. So that's kind of my background, but was always interested in, you know, the future of the industry, if you like. So even back when I was at university in the late 90s, believe it or not, my dissertation was all around, you know, what the industry could look like in 10 years time. And it was interesting to revisit that recently, actually. So I've always been interested in it. But then kind of, you know, your career kind of takes over 20 years into the career. And I got this opportunity within Gleads, who I work for, to move into what was then called the Insight and Analytics Department, which was a great kind of grounding into the world of data and starting to be able to put my interests into practice, which was good. And then eventually that became part of a wider digital team. Speaker 2 (1:47) The AI has become a really important part of that now, the AI deployment, the AI development, the AI training, all those things. All the things kind of marry together really quite nicely. But actually, one of the first, we were experimenting with AI back in about 2016, 2017, well before all the latest hype with generative AI, when we were doing some work with Imperial College, where one of the people that I was working with there was a... a AI professor who let me basically borrow a couple of his undergraduate students and we built our first machine learning model all that time ago which was about 10 years ago and that really kind Speaker 1 (2:26) of Speaker 2 (2:27) yeah Speaker 1 (2:27) and it opened up the floodgates for me in terms of Speaker 2 (2:29) that understanding of the kind of the possibilities that this could mean for our industry and obviously it's taken a while to catch up but then obviously when we started playing around when GPT started coming out, well before chat GPT, it became even more obvious to me that this was going to be a big, big deal for our industry. So, yeah, so now I've got the pleasure of sort of deploying that technology throughout the organization. But then I also do a lot of work for Rix as well, which is, you know, I kind of... a passion of mine to make sure that the industry is moving in the right direction. So for RICS, I've been part of their construction professional group panel, looking at the future of the industry from a construction point of view, and then was part of the working group for the recently released and recently in action responsible use of AI professional guidance note for surveyors. So it's been quite a busy time over the last few years. But what I think is important is, you know, I do understand projects as well. I've been working in projects for many, many years. All the pains of delivering projects as well. Speaker 1 (3:46) What's Project Flux? Speaker 2 (3:48) So that's my kind of labour of love, if you like. I started off as a newsletter because I realised that... I was having to read on the weekends, like just keep up with the insane development in AI. And I thought, well, actually, I'll just turn this into a little newsletter to inform other people, originally on LinkedIn. And it got a lot of interest. So that's turned into quite a well-read newsletter in the industry. There's probably about 6,000 people around the industry subscribed to it now. And then eventually that we ended up doing podcasts as well. We get to interview all kinds of interesting people, authors, academics, practitioners, tech side people. So yeah, you can find that at projectflux.ai. It's free to subscribe and you get newsletters and podcasts. But really the point of it is just to keep people informed, Paul. Speaker 1 (4:42) Yeah. Yeah. So you're at the cutting edge, I imagine. Speaker 1 (4:47) So what we thought we'd talk about is what people can do with, what project managers can do with AI day to day, today. And also then look into the future, what you think, what we think might be happening in the future. And we've had a chat before the podcast, and we're sort of going to put it into three layers. Yeah. So our bottom layer is really the sort of basic use of AI, which is what we've called knowledge assistance. So Speaker 2 (5:17) it's basically Speaker 1 (5:18) using as an encyclopedia Britannica type, Speaker 2 (5:20) you Speaker 1 (5:21) know, load up into the project, the AI machine, everything you know about project management, and it will give you guidance and tell you what the textbooks say and what your procedures in-house say. I've seen quite a lot of people do that. And then we also discussed this next tier up, which we call decision support, which is where you give access to your corporate data systems like planning, finance, drawing status, so that you can begin to create. sort of virtual dashboards or you can begin to query the ar using as an intelligent query tool to find out what's going on in the project and without having to phone people up or talk to them which may or may not be a good thing and then we talked about level three level three is quite interesting that's like organizational learning so that's more working at a portfolio level beginning to look for trends and patterns within the data to see you know um cobs You know, we know why projects fail. Speaker 2 (6:26) Maybe Speaker 1 (6:26) the AI can help us to understand why they fail and catch them before they do fail by giving us indicators. So this is things like benchmarking or real-time benchmarking or lessons learned or double-loop learning. Speaker 1 (6:40) So we thought we'd structure in this way, because I think when you say AI, it's a massive beast to many people and many people interpret it in different ways. Speaker 2 (6:48) They do. And obviously, everyone thinks of it as chat GPT because that's most people's exposure to it. But of course, AI has been around for a long, long time. And the term was coined all the way back in the 50s. So there's a lot of confusion and a lot of people trying to make sense of it. So, yeah, but I think it'd be really good to clarify that for people. Speaker 1 (7:08) So let's talk about sort of level one knowledge assistance. Have you seen that used and what would that mean to you? Well, Speaker 2 (7:19) that's kind of where most people are. I mean, when most people get access to ChatGPT or Copilot, which I imagine most people have got those kind of accesses, the way most people have started is by using it as a glorified Google. So the search terms that we used to use on Google, We've moved to these AIs. And interestingly, there was a study that came out, I think just last week, to show that Google's click-throughs to links has fallen by about 50%, which is significant because that's what their whole business model, their whole AdSense business model was based on people clicking through to links because now they've got something called an AI overview, which is powered by their model, which is called Gemini. So people are already getting very used to... writing a query and then whereas before you'd expect to scroll through a bunch of pages until you found a link that was right for you to now just getting the answer given to you. So we're already at that phase very, very quickly. Yeah, I've seen that Speaker 1 (8:22) with some of the clients I work with. They've got large libraries of procedures and processes and standards and rules and they want people to be able to access those quickly and so they can just... Basically use it as a glorified search tool to find the answers. Yeah, Speaker 2 (8:37) but there's a few problems with that. Number one is the way that we've trained ourselves to use Google for the last sort of two or three decades, which was all around keywords. You know, if you wanted a good result on Google, you had to kind of shorten your prompt for, it wasn't called prompts then, but shorten your search to as few words as possible so that you could narrow in on the thing that you were looking for. The problem is with AI or any of the AI tools, that's a really bad strategy because it's much more agentic. It needs to be told what to do. It needs context. So it actually needs more. information, the more the better. And if you give it very small amounts of information, it's more likely to hallucinate because it always wants to give you an answer. And if you're not giving it the context, it will make up answers. So this is where people have a bad experience at that level one, as you talked about, because they'll use it like Google, expecting to get a better result than Google and sometimes get an actual worse result than Google. So it requires a different mindset, a different way of training. And of course, the other thing is we're really lazy. The good thing about, you know, we used to go to a library and go to find Encyclopedia Britannica and have to read through stuff and find our sources. Then we got used to using Google, but at least we had to do some kind of search and we had to kind of look at the links we were clicking through to and verify those links. And we very quickly moved to a world where we're just given this answer and we're not necessarily testing the citations and the sources, which leads to a lot of the issues that we've heard about. hallucinations and the AI making things up. So level one is the way most people are starting, but a lot of people are getting stuck there because of that reason, because they're not seeing the benefit, because they're not Speaker 1 (10:19) using it in the right way. Yeah. I think this has got real strong applications in constrained bodies of knowledge. So we've had some experience when we're doing exams, which are set on a set text. Speaker 1 (10:35) And so you can define for the model, this is the text to look up the answer in. So you can write system instructions that sit behind what the users see. They say, look up in this book what the answers are. And then you can give it like, these are sample questions that you need to replicate. And this is a prompt, template prompt that we'd like you to use. So basically. Speaker 1 (10:58) What's the word? So the user might put a very short prompt in, but you can tell the AI that actually if they say this, that's what they mean. They mean, you know, give me a long question. So replace that prompt with this longer question. Speaker 2 (11:10) Yeah, and this is where people are basically putting their own body of knowledge on top of. So you've got the harness of the actual model that you're using, whether that's GPT or CLAWD, and then you're layering on your own. information so that you direct it to that information first. That's a really common way of getting started. Lots of people are doing that. It's really easy to do as well. You can do it on a personal point of view. You can do it in Copilot or something like that really easily by creating a project and then giving your information into that project folder. Or you can even create a little mini agent within something like Copilot. So that's a really good, easy first step application. Speaker 1 (11:49) Yeah, nowadays you can just attach a... A folder on Google Drive doesn't allow that. Yeah, yeah. Or SharePoint and say, this is the content I want you to search through. Speaker 2 (11:58) Yeah, yeah. It's getting easier and easier to kind of personalise it. My experience Speaker 1 (12:03) is the level of hallucinations going down. It's getting better at looking at the sources. When you give it a source, it's getting better at scanning those sources. Speaker 2 (12:12) Yeah, it is. And that's all to do with the context windows. Well, it's partly to do with the context windows. It's partly to do with the actual models themselves are getting more powerful. So they understand what they have to do. They run their own checks, their own verification. And it's partly just with us getting more used to it. So we're just getting better at being able to prompt these things as well. So they are getting better. There are still issues, though, and people need to realize that. It's still... You still need a human. I hate the word human in the loop because it implies that we're kind of below the AI. But, you know, that idea of us having some say and not just passing off information that comes out of the AI as our own, because that's what does lead to serious issues. You know, we still got to take control. Speaker 1 (13:01) Before we move on, you said the word context window. Do you just want to expand on what that is? Yeah, Speaker 2 (13:07) so it's the amount of information. So, you know, when these models first started, there was only so much you could put into them. You couldn't even upload documentation or what have you. And the way that these models work is on so-called tokens, which very roughly speaking, when you hear tokens, it's roughly three quarters of a word, right? So when you hear that the context window has gone up to one million tokens, that means you can put in roughly one million words or, you know, three quarters of a million words into it. And that's going up and up all the time. I think the latest model is up to two million context windows. So that's roughly two, you know, whatever that ends up being, around two million words, right? So the more... The bigger context window, the more information it can synthesize and understand. Therefore, the better the response is going to be and the less likely you're going to get hallucinations. In the early days, the context windows were quite short, so it would kind of not maybe read all the information, especially if you uploaded something. The other thing it's got better at is understanding attachments, being able to read images, being able to read tables and PDFs. So basically the rule of thumb, the bigger the context window, the better performance you're going to get. Speaker 1 (14:24) Yeah. So we move on to level two. So level two is what we call decision support. And this is where you basically take the AI as a top layer, but you give it access to call for a number of tools or functions. Speaker 1 (14:42) And we won't need to get into the technology, but call your business systems so it can call in your costs, call in your schedule, it can look at your drawings, you were telling me, status of drawings, status of dashboards. So there's a couple of, you know, there's something called an API, which is an old way of pulling information in, and there's something called an MCP, which is a model context protocol, which means that Speaker 2 (15:09) the Speaker 1 (15:09) AI actually has some... tools that it can pull in. It can say, I want to know about the costs, and it's got a tool that it can pull all the cost data in for that project or tell me about the risks. So it's got a risk register tool that it can pull and update risks. Speaker 1 (15:25) So have you seen people doing this sort of stuff? Speaker 2 (15:28) Yeah. So MCP was a big deal. So the best way of thinking when people talk about MCP is think of it like a USB drive for your AI. So you know when the USB-C came out, it means that you can plug your peripherals into anything. It connects to everything. That was the whole idea around it. There was a standard protocol for connecting your hardware. So the MCP is like your software version of that. It's the way that your AI can talk to anything. Like as soon as a piece of software, let's say, your Microsoft or your Procore or whatever it is that you're using has an MCP available, effectively that can easily then plug into your AI. And then the AI turns from something that talks about your project and something that works inside your project. And those are getting more and more familiar. You can still get connections with APIs, but it's a little bit more fiddly. The MCP is, literally, it turns up as a kind of a switch in ChatGPT or Copilot or Clause, and you just switch it on. And then it's got access to all that stuff. I mean, personally, I use something called Notion, which is like my, where I keep all my notes. There's an MCP. So there's a connection to that to Claude. And now when I talk to Claude, it can see everything in my Notion. I've got the same in my emails. So it's connected to my emails. It's connected to my calendar. So it's a massive game changer because you're now using it. Your AI is becoming an agent. You're turning it into an agent. It's no longer just giving you responses back. It's actually working inside of these. various tools that you've got. And you're going to see a lot more of that. I think it won't be long before it'll be very, very uncommon for any tool not to have an MCP. Speaker 1 (17:07) And have you seen it used for like risk register, risk data, change logs, issue logs, all that sort of thing? Speaker 2 (17:13) Yeah, Speaker 1 (17:13) so... Where you generate Gantt charts, all the data that we generate as project managers. Yeah, so if you take Speaker 2 (17:22) Procore, for instance, they've got an MCP, so if your data's all in this kind of Procore environment, you can connect your MCP to whether it's Claude or whether it's the anthropic model, whether it's Copilot, and then you can start... All that data is interoperable between it. So that's happening right now. That's really common. And the other really simple one is, like I say, emails, like, you know, being able to connect to your emails. So people who are, and this is where the separation is starting to occur. The people who are really getting into the more advanced use of AI are really starting to separate themselves from the people who are just using it as a glorified search engine. Because now those people. have got this agentic capability where the AI is doing stuff for them. And that is not like a 10 % increase in productivity. All of a sudden, they've got maybe 50%, 100 % increase in productivity because they've effectively got agents working for them. So I'll give you a good example. I'll use this call with you. So we had a call the other day to talk about this podcast, what we were going to talk about. We had a transcript of the recording. That was in my Notion. I was able to use my Claude MCP to connect up to the Notion page that was recorded on with some other notes to help me prepare for this particular interview. And I do that with most of my kind of meetings now, not in a kind of weird way, just to help you be prepared. And that just happens automatically. So that's just a really simple example. But a really good example, like for project managers, one of the things that you'll be doing is running from meeting to meeting to meeting. So the ability to kind of have the AI work for you, preparing briefing notes for your meetings, that is eminently doable for any project manager right now. Speaker 1 (19:11) Yeah. And I also think for portfolio managers or program managers, you know, look at my portfolio and tell me what my top 10 risks are across the whole portfolio or what changes are running Speaker 2 (19:22) late. all those sort of questions that I used to run a PMO and all the questions Speaker 1 (19:27) that the director used to ask me and I used to go away and do a massive pivot table. Come back two days later with the answer, you know. It just Speaker 2 (19:34) sucks Speaker 1 (19:35) that data Speaker 2 (19:36) in, really. It