Welcome: AI puts the cloud question back on the table (0:00)
Moises Aguilar: Welcome to our webinar series, The Road to AI. Today's episode is about cloud shopping. I have with me my co-host, Vivek Gopinathlal. Hello, sir. How are you today?
Vivek Gopinathlal: I'm doing well. Thank you.
Moises: The setup for today is for CIOs at companies that haven't jumped on the cloud bandwagon because it didn't make financial sense. It was fine to keep your old systems, and you never found a financial reason to make that kind of investment, that kind of transformation. But then AI came along, and the question is back on the table.
So my first question for my co-host is: how do I do this without breaking the bank? I want to modernize. I want to get ready for AI. But I don't want a huge program with 1,000 people working inside it. I want something small that proves the concept. How do I do this without it becoming a huge endeavor?
Start with an MVP and a clear business use case (1:09)
Vivek: That's the right question. You start with an MVP, a minimum viable product. To get to an MVP, you need the right business use case. From that use case, you start with a small team and build up so that you prove your concept first. You prove your approach and your underlying frameworks, you gain confidence, and then you scale. That should be the way forward.
Copilot or agent? Choosing your first build (1:42)
Moises: For that minimum viable product, do I want to do a copilot? Do I want an agent that automates a task? An agent that takes over a business process? How do I plan, prepare, and design this MVP?
Vivek: Let's take a stand and go with a copilot, as long as it satisfies your business use case. If that's valuable, go for it. Find out whether your use case can be met with a copilot. Once you realize you need something more to scale up or go in a different direction, you build on top of it. But definitely start with whatever is available right now within your platform.
We've seen clients start with a copilot and then come back and say, "I have these use cases, and now I need to scale up for my whole organization, so I need a better architectural solution." That's how it grows. But do start with the simplest option. That's the best idea.
What AI can you trust today? The BTER framework (3:06)
Moises: There are two sides to this conversation. You're telling me to find the business case, but from what I'm reading, agents aren't ready to be trusted right now. I can find a business process I want to automate, or I can say I want to raise the level of knowledge across my finance, legal, or support department with a copilot that assists my employees. The business case is one aspect. But some areas of AI are okay to trust and some we shouldn't touch yet. I need that information. What can't I do right now?
Vivek: That's a very valid question, because we always get lost in it: where should I start and what should I focus on? We use what we call the BTER framework. B stands for business value, T for technical fitness, E for economics, where you try to hold your costs, and R for risk.
Risk decides what gets modernized and how well you govern it. That's the exact question you're asking: can I take this on? That risk evaluation has to happen, and the BTER framework helps you identify the risk and what you can move to the cloud and to AI. You need governance, the right context, and evaluation metrics and frameworks. Then you know what should go into the AI system, and you can scale from there. So there has to be a separate assessment. Start with the BTER framework, get your assessment ready, and then roll it out across your data sets. That's what we generally recommend.
Are there AI projects to avoid right now? (5:28)
Moises: I have so many questions. Question number one: are there AI projects I shouldn't even try right now? Either the technology isn't there, or it's too hard to set up, make work, or trust?
Vivek: There definitely are. There are cases you don't want to get into, and a few you may want to try within your organization, where you're willing to spend resources, make sure you have the right data, and get the right value out of it. It's all driven by value. If there's no business value in it, set it aside. Then there are the cases where the technology doesn't support it yet. Those are engineering problems: can you scale the service and build your system on top of it? You align those factors and decide that certain engineering problems can't be solved for certain use cases yet. The rest comes down to business value. Focus only on the use cases with the right business value, and everyone stays aligned.
The low-hanging fruit: knowledge systems and RAG (6:41)
Moises: I need to deliver something this year, something with the word AI on it, and it has to work. I need the details: what I can't do and what's low-hanging fruit.
Vivek: Let's start with the low-hanging fruit: a knowledge system. A knowledge system answers your questions, so it's easy for business users to start using it and learn what's in it. You put a lot of data into the system for users to consume, and for them to understand it, it has to come back in natural language. A RAG system helps build context around that data and answer in a way people understand. It's a simple, low-hanging fruit that any industry can start with. Even a copilot can help you there, and you can grow from that point.
Moises: Let me make sure I understand this RAG thing with some examples. Imagine I have a large legal department. I take all the regulations we usually work with and make them available to my entire team. Or I have a support team and I want them all to share knowledge. Or I have customer reps and I want them all to have more information about the customers they interact with, so they all offer the same services at the same level. Setting a minimum standard of knowledge and service for a group: is that what RAG is for?
Vivek: To an extent, yes, what you described is right. You provide all your technical and business documentation to the system for it to consume and understand. Whenever users have a question, they ask the system, and it answers in a standard way. It doesn't change, it doesn't hallucinate, and it doesn't give you something different each time.
This solves a typical issue from the traditional days: you need a transition from someone who's leaving the company, or from a senior colleague who isn't available or is busy on another task. With a knowledge system, you query it and get the exact response without it straying from the context. That's RAG: retrieval-augmented generation. It retrieves from the data context it has and then generates an answer users can understand. A RAG system is an excellent starting point.
What shouldn't you touch? That's always the tricky part, because things evolve daily and the pace of change is exponential. Sometimes we tell people, don't start this now, because tomorrow it may be readily available. Or we say, let's start with this one and see where we are in three or four months. When it comes to making decisions, I'd prefer a human in the loop making them rather than the system deciding on its own. It can give you recommendations, but the decision has to be made by a person. That way you have a deterministic system, not a probabilistic one. That's the direction we have to go.
Should AI make decisions? Keeping a person in the loop (10:48)
Moises: If the technology is moving so fast, are we ready for more ambitious solutions? Imagine a supply chain company with many decisions happening at the same time: rerouting, orders coming in, suppliers coming in. Until now, we've had if-then-else programs doing the best they can. Now we have AI that, in theory, makes better decisions. We could have an agent responsible for each area: scheduling, delivery, storage, manufacturing. Are we ready for that?
Vivek: I'd be very careful about saying AI makes better decisions. It makes very good decisions, but "better" is always tricky. The bigger issue is that you can't hold the system responsible. When a person makes a decision, you know what was decided and the rationale behind it. When a machine decides, you need traceability: why did it make that decision? It may be right, and sometimes it will know things we don't, which is perfectly fine. But we need to know the reason. Traceability has to be there for accountability, so you can say, "for these reasons we made this decision," and there should still be an approver.
Take your manufacturing example. Most order management systems involve many clicks to get an order done. AI can learn from past experience and past selections, understand the pattern, and make a similar decision. AI is, in many ways, complex pattern matching. It can quickly find the pattern and give you a good explanation of why it chose what it chose. For certain use cases, we can definitely use AI to make life simpler and focus people only on the steps where you want human validation. That makes it easier for everyone to adopt and embrace AI. You coexist with it; AI doesn't take over every task, and we keep accountability.
One last use case I want to highlight is fraud management. When a transaction comes through, AI can flag whether it looks legitimate or not, but it shouldn't cancel it. A human in the loop has to validate and approve it. You can't let AI simply decline the transaction or let it go through. Those are the areas where, right now, it's very valuable to have a human in the loop.
Moises: Let me summarize what works right now. A conversational agent, this RAG approach you mentioned, is very doable without too much risk. Did I hear that correctly?
Vivek: Yes, that's absolutely correct.
Moises: And if I want to be more ambitious, I put enough guardrails and controls around the agent, and it always gives recommendations. I use the technology to find opportunities people couldn't find on their own, but there's always a suggestion at the end for a person to approve. That's what we should do today.
Vivek: Yes, absolutely. There's no compromise on governance. The human in the loop becomes the governance, so AI is used in a more confined environment and isn't all over the place.
How to deliver an AI win this year on a budget (15:07)
Moises: Now back to my question: how do I do this cheaply this year? That's where I am right now. I have to deliver something this year, I don't want to break the piggy bank, and I want a success on the books. Where's the lowest-hanging fruit, and how do I implement it? What do I buy?
Vivek: It depends on what platform you already have and where you want to go. If you've already decided, we can go that way. Otherwise, the quickest way in is to start with a copilot. And to avoid breaking the piggy bank, pick one use case only. You don't have to boil the ocean. Pick one, pick a team, put an MVP on it, and go from there.
Be careful about bringing your existing systems along by default. It's a 50-50 call. Sometimes the old systems and old data are useful, and sometimes they aren't. You don't always have to take your old baggage with you to your new home. Go in with the mindset of finding the right use case and building from there. Don't start with a platform and then go looking for the problem; that's a hard place to get out of.
Start with the smallest problem. In-house solutions generally work very well first: something that runs within your organization and solves a simple use case, maybe for HR, your developer community, or your legal team. RAG models work well for any of these. Then you start building services that give people live information using agents, and you scale from there. Just make sure you have governance around it.
Modernizing legacy systems with an internal project (17:31)
Moises: Let me give you a use case, since you mentioned the IT department. I want to standardize every document in that group. I want a solution that helps me through the SDLC, making sure every phase is covered and defining the minimum documentation for each phase, with templates. That way every team in my department uses the same formats, phases, and structure.
But my systems are on servers in a data center somewhere, with an Oracle license, and they're 20 years old. That isn't going to fly. Yes, I want AI, but I need the word "modern" around this. If I tell my CxOs I've modernized while I'm still on a Sun server from 2015, that won't fly. I need to show them that technology, processes, and AI are all moving forward. If I pick an internal project with an internal group of people, so there's little risk, how do you make me look good?
Can I trust my data? Moving to a modern platform (19:08)
Vivek: The main thing to get out of this project is the answer to one question: can I trust my data? That should be your core focus. If you can trust your data and build reports on it, your CIOs and CTOs will be very happy, because that's the gap between traditional and modern systems. Every modernization we do strives for that: I can trust the data coming in, and I can trust what's stored in my system. To get there, you start with a platform. If you have a simple, straightforward setup, you can go directly to Snowflake or Databricks, or pick another existing service; there are many more. These cloud services help you pull data into the system and work on it where it lives, without moving it around. That way you can build all your services on top of it.
Sometimes you won't have a business use case at all, and the technical implementation itself is the use case. That's fine. Take the data onto the platform, start small, run it with all the access and services you envision, make sure it works, and go from there. Now your data is on the platform, you've migrated to the cloud or a modernized solution, and you've implemented AI. When you present it to your CIO and CTO, they understand their data and the questions it can answer, and they're happy with the MVP. Then you can start scaling. Governance and economics are very important here, so leadership understands what they're paying for and what they're getting.
Turning scattered project documents into insight (22:00)
Moises: But I don't want my data to be the stopping point either. Take my SDLC solution. For that use case, I'll have project plans, project documentation, status meetings, and presentations on how projects went. Maybe there's no single place with every detail. With all that documentation, whether it's project plans, Word documents, PowerPoints, or status reports, can I bring that knowledge together and create something of value? Tell me which initiatives ran late, where I had too many or too few resources, where I went over budget. That kind of information is very expensive to gather today.
Vivek: Absolutely. Very valid.
Moises: None of that is data in a platform or a database. It's floating around the organization. Can I use it?
Vivek: Absolutely. That's the right starting point, and it's what I meant by in-house MVPs. You bring in all your SOPs: how the process is supposed to work, whether there are mistakes or issues in it, whether everyone follows the same process. That documentation may live in your drive or in Confluence; it could be anywhere. There are services that connect to these sources, pull the information, and let you create a simple RAG model at this level. Confluence also provides its own services for this, and you can still build in-house.
Confluence is a wiki where you store project documents, for the whole company, similar to SharePoint. You can build pages and documentation around your processes, individual project modules, and design documents. Now you can tap into it directly, pull all that information, and make it available to developers, business analysts, the data management team, and the data servicing team. Everyone knows the process that's been built. The system creates its own understanding of what the documents say and serves it up for your purpose.
Or you can build your own RAG system, store it in your vector databases, and use it so you can change it anytime and create multiple versions. The only thing is, you manage it yourself. Service providers generally offer these assistants, so you can use those or build your own. Both work very well. It will definitely help the productivity of the individual contributors in the company, and it's a good start. Again, it depends on the business value you want to bring.
Standard templates and reports from your documentation (25:09)
Moises: I like this option, because let's say I have SharePoint or Confluence and I just have everyone put in what they already have. I'm not asking them to invent anything new. Project plans, status spreadsheets, status presentations, Word documents: every kind of SDLC document goes into this repository. Then I tell it to go through everything and, first, give me templates for each SDLC phase, so the analysts, developers, designers, and testers all work from the same structure. That's really nice. But then, can I run reports? Can it tell me how many projects were late last quarter based on the documentation?
Vivek: Yes. There are tools that give you summarized responses in good charts, so you can visualize it better. There are multiple services available, or you can build your own service that fetches results in the template you want. It will line up all the projects: how many modules, how many were executed successfully, and what issues came up. You provide the template, and it starts reporting in that template.
There are AI services readily available beyond this, too. On the data side, Databricks has an AI service called Genie that can help you scale that way and answer questions quickly. Snowflake has its CoCo models (Cortex platform), where you can have a conversation with your data and run multiple agents. The exact queries you described can be done using these AI services running on the different cloud platforms. We can get it done easily. That shouldn't be a problem.
Recap and parting advice: governance is key (27:50)
Moises: You said easily, which is nice.
Vivek: That's where the world is going. At the end of the day, we evolve to make our lives simpler, so everyone can contribute and use it in a beneficial way. Technology change is always making our lives simpler.
Moises: Thank you very much for your time. You got me thinking. To summarize what I learned today: find a use case. An internal use case is better than going customer-facing, because nobody outside will see it if something isn't working. AI can consume data that isn't just numbers; it can consume more or less anything you throw at it. We can decide how we want that data presented. I can store what the AI produces every month or every week and keep it as history.
There are many opportunities to improve the efficiency of almost any department in my company, whether it's standardizing processes or standardizing the level of knowledge, so knowledge from anywhere in the company is available for everybody to use. You gave me many ideas. Thank you very much. Any parting thoughts before we go?
Vivek: Just remember, governance is the key. Make sure you have the right governance around it and the right data going in, or else AI is going to hallucinate.
Moises: Thank you, Vivek, for your time, and thank you, everybody, for watching. Remember, whatever you build, build for reality. We'll see you in the next one.