home  bbs  files  messages ]

      ZZLI4422             linux.debian.devel             1179 messages      

[ previous | next | reply ]

[ list messages | list forums ]

  Msg # 180 of 1179 on ZZLI4422, Wednesday 9-09-25, 1:11  
  From: AAYUSH RAJ  
  To: ALL  
  Subj: GSoC 2025 Reflections: Building Salsa St  
 From: meet44yu5h@gmail.com 
  
 Greetings everyone! 
  
 I participated in the Google Summer of Code 2025 program with Debian 
 and want to share a recap of what I did this summer. The program 
 officially spanned June to August, and I have been actively 
 contributing to Salsa CI since April. Throughout the summer, I 
 dedicated 450+ hours to development, testing, and collaboration. My 
 main achievement was creating the Salsa Status page for visualizing 
 Salsa CI statistics, which is a result of 160+ commits and 9,500+ 
 lines of code. I also contributed to Salsa CI in various areas, 
 enhancing pipeline speed (caching, auto-cancel) and fixing several 
 issues, and submitted 17 MRs. Overall, the summer was very productive, 
 and I€€€m proud of all that I accomplished. 
  
  
 ## Salsa CI statistics (salsa-status.debian.net) 
 ------------------------------------------------------------- 
  
 The status page was my most important goal and took 3 weeks of 
 planning and prototyping, and most of my time went toward it 
 throughout the summer. It provides both Salsa CI users and the Salsa 
 CI developer team with visibility into CI performance, helping 
 identify wasteful practices, broken configurations, and optimization 
 opportunities across the entire Debian package ecosystem. 
  
 In the last 15 days alone, we have recorded 8500+ pipelines across 
 3500+ projects that have consumed 250,000+ CI minutes. We also spotted 
 55+ projects that haven€€€t been updated in the last 2 or more years, 
 but still run daily/weekly scheduled pipelines that result in large 
 amounts of traffic and unnecessary wastage of resources. We will be 
 reaching out to them soon to resolve this. 
  
 **Features**: The status page aggregates and presents pipeline and job 
 statistics, success rates, performance trends, and detailed CI history 
 with insights across all projects that use Salsa CI. It tracks CI 
 duration trends and resource consumption patterns to optimize 
 performance and identify bottlenecks. The aggregated stats are 
 presented in an organized way and visualized using various charts. 
 Automated notifications for performance degradation are sent on the 
 Matrix room to stay informed of regression and dropped success rate of 
 jobs in real-time. 
  
 **Benefits to Salsa/Debian**: Among several benefits that can€€€t be 
 listed here, the status page helps in early detection of failures, 
 controlled feature rollouts, and post-merge regressions, all while 
 identifying resource wastage of CI resources (multiple long runs, high 
 failure rates). It helps with focused probing of issues using filters 
 and allows developers to compare their project trends over time. The 
 automated Matrix alerts help bring regressions to light in time. 
  
 **The backend**: Located at /api/v1, it provides an API and is 
 responsible for fetching, storing, and serving the data to the 
 frontend. The pipelines using Salsa CI make an API call with pipeline 
 identifiers when the `SALSA_CI_ENABLE_STATS` flag is on. The backend 
 registers the pipeline and inserts their stats and final status into 
 the database with their jobs, average time, result, project details, 
 maintainer, etc. I have also implemented cron jobs to aggregate and 
 insert hourly and daily stats for the pipelines and jobs in the 
 database. This helps serve them quickly and saves the database from 
 repeated strenuous calculations. Apart from this, we have cron jobs 
 for MRs to keep track of whether any merge or feature rollout on Salsa 
 CI pipeline resulted in regression. The backend sends Matrix alerts 
 about regressions to https://matrix.to/#/#salsa-stats:matrix.org 
  
 **The frontend**: Built using React, and the design is inspired from 
 Dribbble. It was a challenge to make the design look good yet keep the 
 implementation simple and easy to understand, but I feel I did a good 
 job. The status page contains mainly 4 pages: 
  
 Home Page - https://salsa-status.debian.net 
 - Header shows global stats for all-time pipelines, jobs, projects, and CI 
 time. 
 - Interactive line chart with customizable durations (24h, 7d, 30d, 
 6m, 1y) visualizing total, passed, and failed pipelines with 
 switchable success/failure rate mode. 
 - MR avatar markers on chart lines (hover for MR details). 
 - Job Types table with descriptions of jobs and 30-day success rate 
 sparklines. 
  
 Pipelines Page - https://salsa-status.debian.net/pipelines 
 - Trend line chart and real-time pipeline grid. 
 - Pipeline cards show ID, Gitlab-style status, timestamps, project 
 details, and Salsa link. 
 - Sortable by status/time/duration and filterable by status, duration 
 ranges, and date ranges. 
  
 Job Types Page - https://salsa-status.debian.net/job-types 
 - Treemap showing job type quantities and success rates by stages 
 (provisioning, build, test). 
 - Grid view with descriptions, sparklines, and 30-day stats (runs, CI 
 minutes, success rate, average duration). 
 - Clicking job types shows a detailed page with line chart and jobs grid. 
  
 Projects Page - https://salsa-status.debian.net/projects 
 - Grid view of all registered projects using Salsa CI. 
 - Cards show version (from pushed tags), last activity, maintainer 
 info (from d/control), and last 30 days stats. 
 - WIP: Project's CI config page. 
  
 The dashboard was initially built to be hosted on Gitlab pages and 
  
 [continued in next message] 
  
 --- SoupGate-Win32 v1.05 
  * Origin: you cannot sedate... all the things you hate (1:229/2) 

[ list messages | list forums | previous | next | reply ]

search for:

328,103 visits
(c) 1994,  bbs@darkrealms.ca