GenAI POC: why 90% will never make it to prod
Three tips to de-risk a GenAI project before writing a single line of code.
According to Forbes, around 90% of GenAI POCs (Proof of Concept) are far from making it to production, or never will. 😬 Why? Because a lot of people dive in head first, carried by the hype, into poorly prepared projects. Teams end up in a dev tunnel that feels like the good old V-cycles, but with one big difference: at the end, it doesn’t work. 💥
So, how do you minimize the risks?
Tip #1: no code at the start, only prompt engineering. On most of my projects, when the doubt is about the LLM’s capability, before even writing a single line of code, I make sure I can produce 80 to 90% of the final result… without code. Ex: with OpenAI and its playground, you can validate a project’s feasibility pretty quickly. It’s as if you’d already spent a month wiring everything up in code, except the code doesn’t even exist yet! A super efficient way to de-risk your project right from the start.
Tip #2: no-code, your friend. An optional but interesting step: use no-code tools to build agent pipelines. You can let business people test the system on a wider variety of test cases to add a bit of heterogeneity to the data. It helps you run into some of the challenges of multi-agent systems.
Tip #3: dev, test and track. Plug your app into your favorite LLM (not sure using frameworks is all that useful). Then, you add automated tests in LLM as Judge mode to validate the results and you set up a tracking system to monitor the trends of your system’s outputs. A winning combo to avoid the missteps hit by the famous 90% of projects that fail.
With these steps, you dramatically increase your chances of shipping a project that really works in production.