One of the JetBrains Research goals is to build the foundational knowledge and tools for a future IDE where AI will converge with new types of multimodal data (e.g., XR, data from wearable technologies or any other kind of sensors) enabling more efficient, focused, and personalized workflows. In other words, at JetBrains Research we are interested in enhancing the human-AI experience by leveraging the potential of the intersection between AI and XR technologies as both sources of rich data input and new ways of the system spatial output.
The “Enhancing Tech Creation through Multimodal Context” internship sits at the intersection of software engineering, machine learning, and cognitive science. During it, you’ll design and build a lightweight VR prototype that leverages eye-tracking data to control and/or enhance the developer experience during coding tasks.
What the topics for internship can look like (examples, not limitations):
A trigger-based model that uses gaze patterns to switch between coding modes (e.g., vibe coding ↔ code review).
Interactive control of a simulated, browser-based coding environment in VR where primary input is gaze (and voice/gestures optional).
A multimodal AI component that incorporates eye-tracking signals to improve inference quality (e.g., relevance ranking, contextual suggestions).
Expected outcomes: Deliver a working prototype, concise technical documentation (architecture, data schema, latency notes), and a brief results memo with next-step recommendations. The prototype will be used for user studies with the potential to be presented at top-tier conferences in the AI-XR field.
Final project direction will be discussed and refined during the interview process.
Strong coding skills (e.g., Python/TypeScript/C#)
Familiarity with ML prototyping
Experience with XR/VR frameworks (e.g., Unity/Unreal/WebXR/OpenXR) or the ability to ramp quickly
Familiarity with data pipelines and basic model training/evaluation
Your responsibilities will include:
Designing and implementing a minimal, modular VR prototype.
Creating and testing a pipeline for working with eye-tracking + interaction data for analysis/ML.
Delivering one focused interaction or model (e.g., mode-switching, gaze-aware navigation, or gaze-guided inference) with a short evaluation.