Text-to-workflow cuts Engineers’ Tedious Task Time to Seconds with Agentic AI Platform

How the language converts to nodes (and back)

To make text-to-workflow possible, Synera’s team first created an interpreter to describe nodes, which are the building blocks of a workflow, and node-based structures using pseudo-python code. The existence of such code made it possible to manipulate nodes with code using LLMs and convert the code back to nodes. Such code is legible for an LLM, so the system can operate using it, making it possible for humans to express intent (in plain language) that is then converted to code, which converts again into nodes seen in the software panel.

Building dataset from scratch

Data scarcity is a challenge in nearly every Artificial Intelligence project. the data may be badly annotated, datasets can be biased, or there may be not enough data to train the solution. In this particular case, there was no existing data at all. But there was a great asset to make use of, a library of thousands of sample workflows. So the idea was that this repository of samples could be turned into a dataset for an LLM to use.

In order to make the system generate workflows using prompts, the process had to be reversed first into a workflow-to-text manner. With that, a viable database was complete, and our team could start building the solution. The dataset was also enriched with completely synthetic data generated by the LLM, so the amount of sample data to learn on was substantial enough to deliver a robust solution.

Making the model work

The team prepared a dataset consisting of prompt, code, and workflow triads, enabling the model to connect each to the other and spot the patterns necessary to deliver text to the workflow builder.

The model works in the other way around to a dataset builder. The Large Language Model inside analyzes the prompt, which is later turned into pseudo-python code representing the user’s demand. Later, the interpreter has to turn the code into the desired workflow, represented as a graph.

An unexpected challenge in this stage came from the diligence of modern LLMs. In some cases, the pseudo-python generated by the LLM was closer to the full version of the programming language, adding functionality which is missing in the pseudo code. This was caused by LLM’s tendency to deliver “as good a job as possible,” where the model delivered better solutions from the programming logic, yet ill-fitted to the required context.

This challenge required the building of a validation mechanism which checks the code before sending it to the interpreter. As a result, the code is sent back and forth between the LLM and validator until working pseudo-python is produced.

RAG system support for less work

With a large database of prompt-code-workflow triads, it became necessary to make the solution more stable and reliable. So our team implemented a Retrieval-Augmented Generation (RAG)-based solution to support the model’s work.

Based on the user’s prompt, RAG searches the database of triads mentioned above, looking for ones as similar as possible. When spotting them, RAG delivers the top three picks to the LLM as “inspiration,” reducing both the chance of a mishap during code generation and the likelihood of misinterpreting the intent of the user.

RAG was also helpful due to the documentation already existing in some workflows, so the system gets an even deeper understanding of what has to be done and why. The system also processes the documentation of the nodes to use in the workflow designer, delivering a necessary context for the LLM to use.

Testing and validation

Last but not least, the system has to be extensively tested and validated in search of mistakes and mishaps. With several components that have to seamlessly operate, the LLM, the interpreter, and the RAG system, there are many points where something could go wrong. That is why the solution is being rolled out gradually, along with tests done by in-house experts and hired specialists alike.“After three months into the POC we started to see the ability to generate complex workflows from a simple text prompt.” – comments Andrew