Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-731

Move weekly Jenkins build to Wednesday nights

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      It has been observed that many tickets are merged just before the weekly build runs on Friday nights (actually 00:00 Pacific Time Saturday). The reviews for such merges may be requested on short notice and may require interactive discussion between the developer and reviewer. The potential interruptions due to this activity seem contrary to the spirit of Focus Friday.

      One way of keeping Friday focused is to move the weekly build to Wednesday night. The weekly build does not take long enough to interfere with developer resources the next day (it typically finishes between 05:00 and 06:00 Pacific). Moving the build also allows any breakages to be debugged and fixed on a workday rather than on a weekend or Focus Friday. It does mean that new containers would now become available (but not necessarily be recommended) for the Rubin Science Platform on Thursdays.

      If including extensive testing in the weekly build does start to infringe on Thursday development resources, then the alternatives would be to allocate more CI resources or to reconsider the run date.

        Attachments

          Issue Links

            Activity

            Hide
            frossie Frossie Economou added a comment -

            Well if we are giving up the opportunity to run long/heavy validation jobs over the weekend, we might as well do the weekly on Wednesdays and make the Science Platform folks less miserable - unmoored from a weekend reference there is nothing special about Thursday.

            Show
            frossie Frossie Economou added a comment - Well if we are giving up the opportunity to run long/heavy validation jobs over the weekend, we might as well do the weekly on Wednesdays and make the Science Platform folks less miserable - unmoored from a weekend reference there is nothing special about Thursday.
            Hide
            jbosch Jim Bosch added a comment - - edited

            I'm fine with Wednesdays (or Tuesdays or Mondays) instead of Thursdays, but don't have a strong preference.  But Sundays (or Saturdays) would be worse for me than Friday.  If I'm working hard the day before/of the weekly tag to get something in, it's because I think it'll help most of the rest of the team to have it in the weekly rather than not have it, and I want to be comfortable asking for help from e.g. reviewers to make that happen.  Working hard the day after the weekly because I broke it is much more rare, but it also seems good to move that off of at least the weekend, and probably quiet day as well, as I know there are people impacted whenever anyone breaks a weekly.

            Show
            jbosch Jim Bosch added a comment - - edited I'm fine with Wednesdays (or Tuesdays or Mondays) instead of Thursdays, but don't have a strong preference.  But Sundays (or Saturdays) would be worse for me than Friday.  If I'm working hard the day before/of the weekly tag to get something in, it's because I think it'll help most of the rest of the team to have it in the weekly rather than not have it, and I want to be comfortable asking for help from e.g. reviewers to make that happen.  Working hard the day after the weekly because I broke it is much more rare, but it also seems good to move that off of at least the weekend, and probably quiet day as well, as I know there are people impacted whenever anyone breaks a weekly.
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment - - edited

            Given Jim's comment, which is also my and probably others' motivation too, I'll just mention that perhaps that would be mitigated somewhat (by reducing the significance of the weekly) by DM-21483 (making the dailies available on lsst-dev).

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - - edited Given Jim's comment, which is also my and probably others' motivation too, I'll just mention that perhaps that would be mitigated somewhat (by reducing the significance of the weekly) by DM-21483  (making the dailies available on lsst-dev).
            Hide
            frossie Frossie Economou added a comment -

            Unfortunately the weekly is fairly significant because it is the basis of the recommended container for nublado users, which is an external facing service. We have been getting by with fairly basic testing since its use is predominantly in-project/friendly user, but as we enter the ComCam era things will get more serious.

            In any case, I am interested in the light of Jim's comments if there are any objections to a midweek (Wed or whatever) build.

            Show
            frossie Frossie Economou added a comment - Unfortunately the weekly is fairly significant because it is the basis of the recommended container for nublado users, which is an external facing service. We have been getting by with fairly basic testing since its use is predominantly in-project/friendly user, but as we enter the ComCam era things will get more serious. In any case, I am interested in the light of Jim's comments if there are any objections to a midweek (Wed or whatever) build.
            Hide
            ktl Kian-Tat Lim added a comment -

            There being no objections to a Wednesday night build, I have retitled the RFC, adjusted the description to match, and will adopt. I will implement today; the first "new" weekly run should be 2020-10-22T00:00:00 PDT.

            Show
            ktl Kian-Tat Lim added a comment - There being no objections to a Wednesday night build, I have retitled the RFC, adjusted the description to match, and will adopt. I will implement today; the first "new" weekly run should be 2020-10-22T00:00:00 PDT.

              People

              Assignee:
              ktl Kian-Tat Lim
              Reporter:
              ktl Kian-Tat Lim
              Watchers:
              Adam Thornton, Eli Rykoff, Frossie Economou, Jim Bosch, John Parejko, Kian-Tat Lim, Krzysztof Findeisen, Meredith Rawls, Merlin Fisher-Levine
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.