# remove dead code and dependencies from datarel

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1
• Sprint:
Science Pipelines DM-S15-4
• Team:
Data Release Production

#### Description

Removing the datarel package entirely has proved to be difficult (DM-2928, DM-2948), so instead I'm simply going to remove non-ingest code (and dead ingest code) from the package, along with its dependencies on ap and testing_endToEnd. Other dependencies will be retained even if they aren't necessary for the code that will remain in datarel, to support lsstDoxygen's use of datarel as a top-level package for documentation generation.

#### Activity

No builds found.
Jim Bosch created issue -
Field Original Value New Value
Link This issue blocks DM-1766 [ DM-1766 ]
 Status To Do [ 10001 ] In Progress [ 3 ]
Hide
Jim Bosch added a comment -

Changes on tickets/DM-2949 of datarel:

 $git --no-pager log --reverse LSST/master..tickets/DM-2949 commit 6e5f88858b38e7001cd27a46dfb8a702ffc46c8a Author: Jim Bosch  Date: Fri Jun 19 15:40:16 2015 -0400    Remove relic pipeline and execution code.    This removes all code related to Stages, the old weekly productions,  and the JobOffice, leaving only ingest and a few utility functions.   commit c82d5d93a4870580b3bb5ef1f601a83d60404622 Author: Jim Bosch  Date: Fri Jun 19 15:43:03 2015 -0400    Remove all ingest code that relies on lsst.ap, and the dependency on ap.    Premise is that the code we'll want to start from when we get ingest working  again is the code that doesn't use ap.   commit fb5cabc0f8c8d879b95b8bd4359c9963a47c73cb Author: Jim Bosch  Date: Fri Jun 19 15:45:21 2015 -0400    Re-enable unit tests for utility functions and ingest.    Both of these tests were already working, and probably should have been  enabled earlier; a comment in the SConscript suggests someone wanted  to just disable all pex_harness-related tests, but got these as well.    The ingest test does require access to a remote server, but I've verified  that it skips the test and issues a warning when the server cannot be  contacted, so I think it's safe.  and lsst_distrib:  $ git show LSST/master..tickets/DM-2949 commit 29f62650a8dd25d7528f4eca44013bab354c539d Author: Jim Bosch  Date: Fri Jun 19 15:55:44 2015 -0400    Remove dependency on testing_endToEnd, which is being retired.   diff --git a/ups/lsst_distrib.table b/ups/lsst_distrib.table index 05389da..60420cb 100644 --- a/ups/lsst_distrib.table +++ b/ups/lsst_distrib.table @@ -4,4 +4,3 @@ setupRequired(ctrl_execute)  setupRequired(ctrl_platform_gordon)  setupRequired(ctrl_platform_lsst)  setupRequired(datarel) -setupRequired(testing_endtoend) 

Show
Jim Bosch added a comment - Changes on tickets/ DM-2949 of datarel: $git --no-pager log --reverse LSST/master..tickets/DM-2949 commit 6e5f88858b38e7001cd27a46dfb8a702ffc46c8a Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Fri Jun 19 15:40:16 2015 -0400 Remove relic pipeline and execution code. This removes all code related to Stages, the old weekly productions, and the JobOffice, leaving only ingest and a few utility functions. commit c82d5d93a4870580b3bb5ef1f601a83d60404622 Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Fri Jun 19 15:43:03 2015 -0400 Remove all ingest code that relies on lsst.ap, and the dependency on ap. Premise is that the code we'll want to start from when we get ingest working again is the code that doesn't use ap. commit fb5cabc0f8c8d879b95b8bd4359c9963a47c73cb Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Fri Jun 19 15:45:21 2015 -0400 Re-enable unit tests for utility functions and ingest. Both of these tests were already working, and probably should have been enabled earlier; a comment in the SConscript suggests someone wanted to just disable all pex_harness-related tests, but got these as well. The ingest test does require access to a remote server, but I've verified that it skips the test and issues a warning when the server cannot be contacted, so I think it's safe. and lsst_distrib:$ git show LSST/master..tickets/DM-2949 commit 29f62650a8dd25d7528f4eca44013bab354c539d Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Fri Jun 19 15:55:44 2015 -0400   Remove dependency on testing_endToEnd, which is being retired.   diff --git a/ups/lsst_distrib.table b/ups/lsst_distrib.table index 05389da..60420cb 100644 --- a/ups/lsst_distrib.table +++ b/ups/lsst_distrib.table @@ -4,4 +4,3 @@ setupRequired(ctrl_execute) setupRequired(ctrl_platform_gordon) setupRequired(ctrl_platform_lsst) setupRequired(datarel) -setupRequired(testing_endtoend)
 Reviewers Kian-Tat Lim [ ktl ] Status In Progress [ 3 ] In Review [ 10004 ]
Hide
Jim Bosch added a comment -
Show
Jim Bosch added a comment - Buildbot run successful: http://lsst-buildx.ncsa.illinois.edu:8010/builders/DM_stack/builds/7224
Hide
Kian-Tat Lim added a comment -

Does it make sense to change the name of what's now left after this ticket to daf_ingest and replace datarel with a new package that just has doc and ups subdirectories (and, I guess, an SConstruct)?

Show
Kian-Tat Lim added a comment - Does it make sense to change the name of what's now left after this ticket to daf_ingest and replace datarel with a new package that just has doc and ups subdirectories (and, I guess, an SConstruct )?
Hide
Kian-Tat Lim added a comment -

The removals all look appropriate to me, as does what is left behind. Sad to see some of that stuff go, particularly when we don't necessarily have good replacements for all of it yet...

Show
Kian-Tat Lim added a comment - The removals all look appropriate to me, as does what is left behind. Sad to see some of that stuff go, particularly when we don't necessarily have good replacements for all of it yet...
 Status In Review [ 10004 ] Reviewed [ 10101 ]
Hide
Jim Bosch added a comment -

I don't think it makes sense to change the name yet, as we're also still using this package as a metapackage for the cross-linked Doxygen build (see DM-2948). That means we're still carrying more dependencies than the ingest code here needs, and we can't change the name without changing the buildbot scripts (which I can't do without help from SQuaRE, and they're busy).

Show
Jim Bosch added a comment - I don't think it makes sense to change the name yet, as we're also still using this package as a metapackage for the cross-linked Doxygen build (see DM-2948 ). That means we're still carrying more dependencies than the ingest code here needs, and we can't change the name without changing the buildbot scripts (which I can't do without help from SQuaRE, and they're busy).
Hide
Jim Bosch added a comment -

Merged to master.

Show
Jim Bosch added a comment - Merged to master.
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]
Hide
Kian-Tat Lim added a comment -

Jim Bosch You missed the part about adding a new (nearly-empty) datarel that would serve as the metapackage for the cross-linked Doxygen build and the buildbot scripts.

Show
Kian-Tat Lim added a comment - Jim Bosch You missed the part about adding a new (nearly-empty) datarel that would serve as the metapackage for the cross-linked Doxygen build and the buildbot scripts.
Hide
Jim Bosch added a comment -

Oh, I did miss that part somehow, reading too fast - sorry about that. I'm still not sure it's worthwhile to do right now, as I think a better solution for the Doxygen/buildbot problem is to remove the need for a single top-level package. But we may want to do that first if Jacek Becla's team starts working on updating the ingest code before DM-2948 is addressed.

Show
Jim Bosch added a comment - Oh, I did miss that part somehow, reading too fast - sorry about that. I'm still not sure it's worthwhile to do right now, as I think a better solution for the Doxygen/buildbot problem is to remove the need for a single top-level package. But we may want to do that first if Jacek Becla 's team starts working on updating the ingest code before DM-2948 is addressed.
 Epic Link DM-1769 [ 15588 ]
Hide
John Swinbank added a comment -

Adding this to DM-1769 since it was necessary to unblock DM-1766.

Show
John Swinbank added a comment - Adding this to DM-1769 since it was necessary to unblock DM-1766 .
 Link This issue is duplicated by DM-305 [ DM-305 ]

#### People

Assignee:
Jim Bosch
Reporter:
Jim Bosch
Reviewers:
Kian-Tat Lim
Watchers:
Jim Bosch, John Swinbank, Kian-Tat Lim