The Stellar Forge: Automating astronomical scale CI-CD via Jenkins

Presented by: Ahmed Mubbashir Khan, ESO

The VLT Data Flow System, critical for ESO operations and operational for over 20 years, relies on dozens of interacting tools and ‘nebulae of code’, for multiple telescopes, dozens of instruments and hundred of heterogeneous components overtime engineers have used different systems for building and packing software slowly things started converging, Jenkins came into the picture and forge started.

This is an experience report on how we are using Jenkins to build, test (execution), publish and deploy (part of the system) from Heterogeneous build systems to Jenkinsfile and Jenkins Shared Libraries, from copying artifacts to deploying to production on demand transforming our fragmented meteoroid stream into a synchronous satellite constellation.

This Automation and optimization of build, test (execution) and related activities in continuous integration, delivery, and deployment workflows have greatly improved our performance in such activities. And we are still evolving, creating a new spiral arm of automation within our galactic system, constantly refining its structure and velocity.