The Software Development Life Cycle has evolved dramatically over recent years, with DevOps emerging as a critical element. It’s estimated to reach a market size of $25 billion by 2028, and a recent survey suggests that it commands 80% of engineering budgets, according to Grove Ventures.
Yet, despite its significance, the practice is still considerably complex. Balancing speed with quality, orchestrating multi-layered deployments, and maintaining rigorous security standards are just a few of the hurdles encountered. Amidst these complexities, Generative AI is making a profound impact.
Within this dynamic landscape, Kubiya, an Israeli startup, has swiftly capitalized on the potential of Generative AI for DevOps. Founded in 2022 by Amit Govrin and Shaked Askayo, and with $6 million in their coffers, Kubiya developed an LLM-powered DevOps assistant ‘Kubi’, offering a comprehensive suite of engineering tools, enabling tasks such as CI jobs, Kubernetes namespace management, and Jira ticket handling with unprecedented ease.
In my ongoing series of interviews with pioneers in Generative AI, I recently sat down with Govrin. We discussed their groundbreaking work with ‘Agents’, his early bets on Conversational AI and their ambitions to push the envelope further.
How has Generative AI helped in the implementation of DevOps?
“Generative AI gave way to a more efficient agent framework. The traditional one-sided human to machine interaction, which lacked any form of feedback and required substantial user effort, has evolved into a reciprocal communication medium. It allows for more intuitive and interactive experiences, significantly enhancing the way humans interact with machines.”
Clarify the magnitude of the progression from using Conversational AI to Generative AI. How dramatic was that?
“Older Chatbot frameworks like Rasa Ross were intent-based frameworks, an older version of Conversational AI. They were fraught with challenges, weak in predicting user conversations and they required extensive training. It was ultimately a poor user experience and costly.”
“Where the evolution of Large Language Models took place is that now, for the first time, these LLMs are able to generate a lot of these conversational elements… and fill in the parameters of the conversation on behalf of the user. Interaction is more natural and less rigid. By virtue of fine-tuning the LLMs… we’re able to get much more accurate results in much faster response time by building these agents and frameworks around it. Each one of these subtasks would be correlated to a purpose-built LLM.”
“We built layers of agents in our system to address the specific needs in enterprise environments. The first interaction is with the retrieval of knowledge, general purpose knowledge or company domain specific knowledge, which employs a RAG technique in order to train on that data. But not every developer has permission to take on any of these tasks or requests that they have. So we have a policy agent that grants somebody access to that request. The next level is a data masking agent, masking critical data in the event that somebody without permission asks for a Jira board, views those tickets, or tries to access a restricted Confluence page. Altogether, they ensure data security and policy management.”
Which LLMs are you using? And how has that changed since you started using them?
“We started off in the early beta programs of GPT-2 and GPT-3.5. Now, we’re using a range of models between Hugging Face, Mistral, Cohere, Anthropic, and Azure GPT. We’re actually using the privately hosted version of Azure GPT. We’re also using Llama. Some of them are open source that we’re fine tuning ourselves and some of them are fine-tuned models by public providers.”
“One of the areas that we’ve developed internally is that we’ve been able to decouple our experience from the LLM itself. We’re using the concept of approved models similar to what Amazon Bedrock is doing. We’re curating these data sets in order for the user to be able to go and to bring in their own data, without concern about who’s hosting your LLM.”
What are some of the latest features you’ve been able to add thanks to upgrades?
“The agents have been our breakthrough. It’s a GPT that’s purpose-built for DevOps. People can bring in their own tools, scripts, integrations and interact with the system naturally and almost instantaneously, with very little configuration needed. You could actually be up and running within a few minutes in Slack with Kubiya.”
“The concepts we’re bringing include permissioning and using natural language to grant permission and authorize requests. Being able to create terraform modules using natural language; everything that DevOps would encounter on a day-to-day basis you’re able to do very easily with a simple prompt. If you can imagine the biggest challenge of the early day chatbots to automate, this has been completely removed. And being powered by LLMs, you can just ask it a simple sentence and schedule a task, schedule a job or ask it to do something and it does it instantaneously without needing to preconfigure any automation in advance.”
Can you elaborate about user engagements. How’s that been changing as you’ve added more and more features and become more advanced?
“A lot of our developments are user feedback and user generated. And the features that I described earlier were our early users initially needing to build their business case to their C Suite, like managing Kubernetes and GitHub actions.”
“We have upwards of 20-25 registrations per day, some of them are going into side projects and experimentations, while others are signing up for POCs. Some go on to solve organizational wide challenges that they have using Kubiya.”
In the DevOps and Generative AI for DevOps market, how does Kubiya stand apart from competitors, and how do you distinguish yourself?
“We’ve married two distinct worlds together: one is a world of platform engineering of internal developer platforms with the world of advanced virtual assistance. We’ve layered in Generative AI and Conversational AI throughout the entire domain of internal developer platforms. It completely changed the way people interact with their tools. Mind you, this is the bet that we took prior to the debut of ChatGPT; that conversation will dominate the way people want to consume the various organizational processes and tools, especially for DevOps.”
“We do see competitors in parentheses because nobody up until this point has come forward with a one-to-one type of thesis that we’ve done. Luckily, we’ve had the advantage of a head start in this case, but we need to keep our momentum. Most people are currently still looking at Devtools, and we said that’s the wrong way of looking at it. People don’t want to have another tool, another portal to rely on. They want to live on the platforms they currently interact with on a day-to-day basis. We flipped the thesis on its head and said we’ll bring everything to you instead of you needing to go to your tools.”
If the aim of what you’re doing is to improve productivity of developers, can you put any kind of measure on that?
“Efficiency and time-saving gains. It is still a fairly naive way of measuring it but we attribute a number of efforts to every type of action. Every time somebody triggers some type of bot interaction, we’re able to measure that and add that up over time. A given organization can easily see dozens, if not hundreds of hours per month that they’re saving, which is equivalent to headcount reduction.”
“Whether you’re replacing human beings, which is not the ideal output, but otherwise gaining more productivity out of that same human, here’s a type of output that you’re receiving as a result of using Kubiya. When you’re seeing hundreds of hours saved a month in a fairly small-scale organization…you could see two, three, four (one of our customers showed close to five) headcounts saved in a single month while using the system.”
What happens to the developer in future and if they’re using things like this does this mean there’s going to be less of them? How do you see the developer role changing?
“Just like GitHub’s Copilot is a human operator for developer productivity, so is code generation with a human-paired operations assistant. Many of these people and the people who embrace it will be even more valuable for the organization.”
“The human displacement aspect of it is typically the lower level workers who don’t adapt to the changing times. Those types of processes and things that are currently being done by humans are very likely going to be replaced by automation and machines over time.
“Unless these people adapt to the changing times, their roles are going to be deprecated over time, regardless of what the tool of choice is. This is the Industrial Revolution all over again. It didn’t displace the humans. It just allowed everybody to specialize in a specific area. Rather than having one person doing everything, now you have specialized people who are not being displaced by machines, but are being empowered to be domain experts. A very effective prompt engineer and be even more valuable to their company if they do it right.”
Looking ahead, what’s in store for the immediate future, and the long term roadmap of Kubiya?
“We see a huge opportunity to expand the internal developer platforms to even more domain-specific areas, like SecOps, DevSecOps, and FinOps. We’re not necessarily replacing the various tools. We’re providing a very easy way to automate and to augment, using our system.”
“I think there will be a day, not too far into the distance, where we’ll be using autonomous agents to do a lot of our day-to-day work. We’re enablers about training on data, purpose-built data, from different organizations. We’re not training on customer data. We’re looking to train the world’s first autonomous operators where you could hire your DevOps, using an autonomous agent like Kubiya.”
