Decide how to handle phone-specific syncs from Debian unstable to PureOS
Incomplete, WishlistPublic

Description

Hi,
while we currently block migration of libhandy to Debian testing until we have more API stabilized it would be great to have it in PureOS already. People start to develop for the Librem5 and we recommend PureOS for that. If they want to work on GtkBuilder ui files or read the documentation offline (or build a program locally) they need these packages. Is it possible to sync libhandy from unstable automatically?
I'll ping again once we unblock testing migration so this extra rule can be removed.

guido created this task.Oct 10 2018, 7:44 AM
jonas.smedegaard added a subscriber: jonas.smedegaard.

Only Matthias has access to automation tools, I believe.

mak added a comment.Oct 12 2018, 12:20 PM

We don't automatically sync stuff with Debian unstable, and if I do a one-time sync the package will be out of date again rather soon.
So, maybe just uploading the latest version of libhandy to PureOS directly is a better option for you? It already is in the laboratory/purple suites afterall.

Uploading to once place is more comfortable than to two ;)

Seriously what I'm trying to find out is what we can do and what we can't atm. While I can handle this single case for libhandy during Debian's freeze there might be multiple occasions where we want to stay in sync with packages in unstable or experimental (e.g. for newer GNOME) and we will be having a hard time to do that by hand so being able to sync from other suites would be more than a nice to have feature.

I'll upload a package to green then for the time being.

mak added a comment.Oct 12 2018, 1:27 PM

The problem here is that we sync binaries and are based on Debian testing. We can't just selectively sync some stuff from unstable, because unstable packages might be built against other things that are only available in unstable (e.g. newer versions of libraries with soname bump). So doing that properly would always be a manual action.
I could change the code and implement something that allows syncing some packages only by source, but that will take a bit of time and might likewise cause trouble with packages synced from testing that would have to be rebuilt against the source-sync from unstable. That's why I did not implement this feature initially, in principle, to avoid having scenarios where the automation breaks things or causes unexpected sideeffects.

With a manual upload, you know exactly what is in the suite and, more importantly, when a certain package is uploaded. By uploading to PureOS you can also iterate faster for new packages than going through Debian (so things can mature in PureOS and can then be uploaded to Debian, as opposed to pinning them in unstable with a RC bug (where I wonder why these packages weren't instead uploaded to experimental...))

So, tl;dr I am not strictly against implementing something that does selective source sync from a suite other than the one we are already syncing with, but I see risks in doing something like that. Furthermore you will get a delay from packages uploaded to Debian to them appearning in PureOS, which - when you develop on PureOS primarily - is likely unwanted.

guido added a comment.Oct 12 2018, 2:21 PM

You have me confused now. I thought the whole transitioning between landing -> and green is there to detect these inconsistencies?
Also I'd be syncing for me does not imply syncing binaries. Rebuilding the ones not synced from testing would be a great first step in rebuilding everything.

I'm all open for discussion here (that's why I started this after all).

mak added a comment.Oct 12 2018, 2:30 PM

You have me confused now. I thought the whole transitioning between landing -> and green is there to detect these inconsistencies?

Yes, but first not all of these issues are actually detected, there are always some (rare) cases that fall through the cracks, and if there are these issues, a lot of packages suddenly get stuck in landing and we have to manually sort out the packages, which is usually not what you want with automation.
It does catch most of the issues though before they actually reach any end user, that is true.

Rebuilding the ones not synced from testing would be a great first step in rebuilding everything.

That's actually a necessity, if we would do that.

At the moment I wonder though whether you would be better off in controlling what reaches PureOS directly, or maybe even add a hook to the CI to upload phone source packages into laboratory if the CI succeeded.

guido added a comment.Oct 12 2018, 2:38 PM

Please see the GNOME case above, this is really not tied to our CI.

For our CI we can (and will auto upload to purple/laboratory for some stuff but I'm really talking about things we want from other Debian suites not necessarily maintained by us.

Anything we can do to make developing apps for the Librem phone smoother is really important. Does it make sense at this point to author a strategy / procedures for packages like this? Maybe a separate repo for unstable packages that we want to make available on PureOS as quickly as possible to encourage Librem phone development? I'm still getting my feet wet, so perhaps some of this is worked out already.

It's going to be a central part of our strategy in the next few months to get devs to build apps/port apps for the Librem5.

We have purple and laboratory in PureOS and the CI repos for phone development.

This issues is about automating the tasks at hand so we can deliver good quality constantly.

@guido got it, thanks! The further we can think down the road the better, so if I can be of any help let me know. I'm not yet sure what else might be necessary / desired by devs for mobile app development.

guido added a comment.Oct 13 2018, 1:58 PM

@sean.obrien (this might become a bit off topic): the flatpak side needs more work for the 3rd party apps.

In T594#11055, @guido wrote:

@sean.obrien (this might become a bit off topic): the flatpak side needs more work for the 3rd party apps.

I am a bit worried about flatpak in general (e.g. http://flatkill.org ) My opinion (but I can be convinced otherwise) is to use APT wherever appropriate third-party repos are available, or to package ourselves if it's feasible.

There are some questions about FSF FSDG in this regard, which @jonas.smedegaard and I have already gone back and forth on, but I do think the conversation needs to happen eventually.

So yes @guido it's off-topic here... where can and should it be discussed? I for one will be trying to get some of the popular E2EE apps in PureOS userland, if we can do it.

mak added a comment.Oct 16 2018, 11:51 AM

@guido

For our CI we can (and will auto upload to purple/laboratory for some stuff but I'm really talking about things we want from other Debian suites not necessarily maintained by us.

A fast way to work around this would be to upload these particular packages to PureOS explicitly as well.
What we can not do is sync things from unstable into landing automatically, as landing is both testing based as well as something that our end users will eventually use. Pulling in bits from unstable immediately before they have even been tested by Debian is something I would consider way too risky.

Pulling a dedicated set of packages from a non-testing suite into the laboratory suite is something we could consider, however. This will require some manual work though, because an explicit pull of a package from a source suite that the target suite isn't already based on means that some necessary dependencies may be missing, rendering the package unbuildable.
So someone would have to monitor that situation a bit and upload missing dependencies.
In Tanglu, we once had a system which resolved these things automatically, but - due to extensive dependency chains - this often ended up in a massive amount of packages being pulled from a suite, which we really should avoid to not make debugging unnecessarily hard.

So, in general automatic syncs from unstable to landing is something I would object to, automatic syncs from unstable to laboratory is something to consider, and explicit package uploads are already possible. In the latter cases, some manual monitoring is useful (as it is in general, having an overview on what is going on in each suite and what their health status is is of paramount importance when maintaining a distribution)

mak added a comment.Oct 16 2018, 11:54 AM

@sean.obrien While that "Flatkill" (what a dumb name...) makes excellently valid points, there's nothing that is a fundamental flaw in Flatpak's design that won't be resolved with better maintenance and more development time. However, that's something that still needs to happen, and when creating something new, you need to start somewhere ;-)
But anyway, it's better to have this discussion at a different place, to not make the discussion on this (syncing) issue overly hard to follow.

guido added a project: Restricted Project.Oct 17 2018, 12:43 PM
mak changed the task status from "Open" to "Incomplete".Oct 21 2018, 1:00 AM
mak triaged this task as "Wishlist" priority.
mak changed the title from "Please sync libhandy from Debian unstable" to "Decide how to handle phone-specific syncs from Debian unstable to PureOS".
mak added a project: Restricted Project.

Add Comment