
| Msg # 180 of 1179 on ZZLI4422, Wednesday 9-09-25, 1:11 |
| From: AAYUSH RAJ |
| To: ALL |
| Subj: GSoC 2025 Reflections: Building Salsa St |
From: meet44yu5h@gmail.com Greetings everyone! I participated in the Google Summer of Code 2025 program with Debian and want to share a recap of what I did this summer. The program officially spanned June to August, and I have been actively contributing to Salsa CI since April. Throughout the summer, I dedicated 450+ hours to development, testing, and collaboration. My main achievement was creating the Salsa Status page for visualizing Salsa CI statistics, which is a result of 160+ commits and 9,500+ lines of code. I also contributed to Salsa CI in various areas, enhancing pipeline speed (caching, auto-cancel) and fixing several issues, and submitted 17 MRs. Overall, the summer was very productive, and I€€€m proud of all that I accomplished. ## Salsa CI statistics (salsa-status.debian.net) ------------------------------------------------------------- The status page was my most important goal and took 3 weeks of planning and prototyping, and most of my time went toward it throughout the summer. It provides both Salsa CI users and the Salsa CI developer team with visibility into CI performance, helping identify wasteful practices, broken configurations, and optimization opportunities across the entire Debian package ecosystem. In the last 15 days alone, we have recorded 8500+ pipelines across 3500+ projects that have consumed 250,000+ CI minutes. We also spotted 55+ projects that haven€€€t been updated in the last 2 or more years, but still run daily/weekly scheduled pipelines that result in large amounts of traffic and unnecessary wastage of resources. We will be reaching out to them soon to resolve this. **Features**: The status page aggregates and presents pipeline and job statistics, success rates, performance trends, and detailed CI history with insights across all projects that use Salsa CI. It tracks CI duration trends and resource consumption patterns to optimize performance and identify bottlenecks. The aggregated stats are presented in an organized way and visualized using various charts. Automated notifications for performance degradation are sent on the Matrix room to stay informed of regression and dropped success rate of jobs in real-time. **Benefits to Salsa/Debian**: Among several benefits that can€€€t be listed here, the status page helps in early detection of failures, controlled feature rollouts, and post-merge regressions, all while identifying resource wastage of CI resources (multiple long runs, high failure rates). It helps with focused probing of issues using filters and allows developers to compare their project trends over time. The automated Matrix alerts help bring regressions to light in time. **The backend**: Located at /api/v1, it provides an API and is responsible for fetching, storing, and serving the data to the frontend. The pipelines using Salsa CI make an API call with pipeline identifiers when the `SALSA_CI_ENABLE_STATS` flag is on. The backend registers the pipeline and inserts their stats and final status into the database with their jobs, average time, result, project details, maintainer, etc. I have also implemented cron jobs to aggregate and insert hourly and daily stats for the pipelines and jobs in the database. This helps serve them quickly and saves the database from repeated strenuous calculations. Apart from this, we have cron jobs for MRs to keep track of whether any merge or feature rollout on Salsa CI pipeline resulted in regression. The backend sends Matrix alerts about regressions to https://matrix.to/#/#salsa-stats:matrix.org **The frontend**: Built using React, and the design is inspired from Dribbble. It was a challenge to make the design look good yet keep the implementation simple and easy to understand, but I feel I did a good job. The status page contains mainly 4 pages: Home Page - https://salsa-status.debian.net - Header shows global stats for all-time pipelines, jobs, projects, and CI time. - Interactive line chart with customizable durations (24h, 7d, 30d, 6m, 1y) visualizing total, passed, and failed pipelines with switchable success/failure rate mode. - MR avatar markers on chart lines (hover for MR details). - Job Types table with descriptions of jobs and 30-day success rate sparklines. Pipelines Page - https://salsa-status.debian.net/pipelines - Trend line chart and real-time pipeline grid. - Pipeline cards show ID, Gitlab-style status, timestamps, project details, and Salsa link. - Sortable by status/time/duration and filterable by status, duration ranges, and date ranges. Job Types Page - https://salsa-status.debian.net/job-types - Treemap showing job type quantities and success rates by stages (provisioning, build, test). - Grid view with descriptions, sparklines, and 30-day stats (runs, CI minutes, success rate, average duration). - Clicking job types shows a detailed page with line chart and jobs grid. Projects Page - https://salsa-status.debian.net/projects - Grid view of all registered projects using Salsa CI. - Cards show version (from pushed tags), last activity, maintainer info (from d/control), and last 30 days stats. - WIP: Project's CI config page. The dashboard was initially built to be hosted on Gitlab pages and [continued in next message] --- SoupGate-Win32 v1.05 * Origin: you cannot sedate... all the things you hate (1:229/2) |
328,103 visits
(c) 1994, bbs@darkrealms.ca