Problem:
YouTrack provides a powerful Workflow API in JS that lets users customize and automate the lifecycle of issues. Although, the API and JS language itself have a noticeable entry threshold which prevents non-dev users from adding such workflows.
Project goals:
Collect dataset and implement an evaluation pipeline for workflow generator AI Agent: The eval pipelines should help to track the agent accuracy improvement and validate the agent output.
Implement a robust workflow generator AI Agent: The generator will translate user-defined rules written in natural language into valid YouTrack Workflow API JavaScript code.
Integrate with YouTrack (optional, if time permits): To generate workflows right in YouTrack, test them and apply to projects.
Outcome:
An AI agent prototype which receives a user prompt (expected workflow rule behaviour) as input and returns the workflow rule in JS as output. The output accuracy should be proven by the evaluation pipeline based on a diverse dataset of input-outputs.
Tech stack:
Kotlin + Koog for the agent development (OpenAI or Anthropic LLMs)
JS, GraalJS, YT Workflow JS API for the workflow development
Langfuse, AI Dev Kit (internal), TeamCity for the evaluation
Good Kotlin and JS knowledge
Understanding how to properly prompt LLMs and how AI agents work
Willingness to dive deep into complex edge cases and propose robust solutions