gitlab.fd.o financial situation and impact on services

[Carsten Haitzler]

On Fri, 28 Feb 2020 08:49:36 +0100 Daniel Vetter <daniel.vetter at ffwll.ch> said:

> On Fri, Feb 28, 2020 at 2:54 AM Carsten Haitzler <raster at rasterman.com> wrote:
> >
> > On Thu, 27 Feb 2020 22:27:04 +0100 Daniel Vetter <daniel.vetter at ffwll.ch>
> > said:
> >
> > Might I suggest that given the kind of expenses detailed here, literally
> > buying 1 - 4 reasonably specced boxes and hosting them at OSUOSL would be
> > incredibly cheaper? (we (enlightenment.org) have been doing so for years on
> > a single box). We farm out CI to travis via gihub mirrors as it's not
> > considered an essential core service (unlike mailing lists, git,
> > phabricator whch nwe still run - we can live without CI for a while and
> > find other ways).
> >
> > The cost is the odd HDD replacement every few years and maybe every 10y or
> > so a new box. That's a massively lower cost than you are quoting below.
> >
> > OSUOSL provide bandwidth, power, rack space etc. for free. They have been
> > fantastic IMHO and the whole "no fat bills" is awesome and you get a full
> > system to set up any way you like. You just bring the box. That should drop
> > cost through the floor. It will require some setup and admin though.
> 
> We've moved away from this setup (all other fd.o services run like
> this) because in a volunteer run org with volunteer admins, admin time
> is a premium. So if we'd go the physical boxes route we could indeed
> cut a lot of hosting costs. But we'd need to get a paid admin from
> somewhere to keep the lights on. Aside from the money needed this has
> the problem that X.org has zero experience with contractors, and SPI
> as our fiscal sponsor also only just started with one part time
> contractor.
> -Daniel

for the kind of money you are mentioning a "1-2 day per week" contractor might
work. we had a volunteer admin and it didn't work out due to them simply not
having the time anymore so i understand the problem. it's why i moved to
simplify setup so it's maintainable with much less effort. :)

> > > Hi all,
> > >
> > > You might have read the short take in the X.org board meeting minutes
> > > already, here's the long version.
> > >
> > > The good news: gitlab.fd.o has become very popular with our
> > > communities, and is used extensively. This especially includes all the
> > > CI integration. Modern development process and tooling, yay!
> > >
> > > The bad news: The cost in growth has also been tremendous, and it's
> > > breaking our bank account. With reasonable estimates for continued
> > > growth we're expecting hosting expenses totalling 75k USD this year,
> > > and 90k USD next year. With the current sponsors we've set up we can't
> > > sustain that. We estimate that hosting expenses for gitlab.fd.o
> > > without any of the CI features enabled would total 30k USD, which is
> > > within X.org's ability to support through various sponsorships, mostly
> > > through XDC.
> > >
> > > Note that X.org does no longer sponsor any CI runners themselves,
> > > we've stopped that. The huge additional expenses are all just in
> > > storing and serving build artifacts and images to outside CI runners
> > > sponsored by various companies. A related topic is that with the
> > > growth in fd.o it's becoming infeasible to maintain it all on
> > > volunteer admin time. X.org is therefore also looking for admin
> > > sponsorship, at least medium term.
> > >
> > > Assuming that we want cash flow reserves for one year of gitlab.fd.o
> > > (without CI support) and a trimmed XDC and assuming no sponsor payment
> > > meanwhile, we'd have to cut CI services somewhere between May and June
> > > this year. The board is of course working on acquiring sponsors, but
> > > filling a shortfall of this magnitude is neither easy nor quick work,
> > > and we therefore decided to give an early warning as soon as possible.
> > > Any help in finding sponsors for fd.o is very much appreciated.
> > >
> > > Thanks, Daniel
> > > --
> > > Daniel Vetter
> > > Software Engineer, Intel Corporation
> > > +41 (0) 79 365 57 48 - http://blog.ffwll.ch
> > > _______________________________________________
> > > xorg-devel at lists.x.org: X.Org development
> > > Archives: http://lists.x.org/archives/xorg-devel
> > > Info: https://lists.x.org/mailman/listinfo/xorg-devel
> > >
> >
> >
> > --
> > ------------- Codito, ergo sum - "I code, therefore I am" --------------
> > Carsten Haitzler - raster at rasterman.com
> >
> 
> 
> -- 
> Daniel Vetter
> Software Engineer, Intel Corporation
> +41 (0) 79 365 57 48 - http://blog.ffwll.ch
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: https://lists.x.org/mailman/listinfo/xorg-devel
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
Carsten Haitzler - raster at rasterman.com