# Please do not strip {{tests/}} from Pipelines Docker images

XMLWordPrintable

## Details

• Type: Story
• Status: To Do
• Resolution: Unresolved
• Fix Version/s: None
• Component/s:
• Labels:
None
• Templates:
• Team:
SQuaRE

## Description

The lsstsqre/centos Docker images are explicitly constructed without the tests directory.

Unfortunately, the tests for some packages rely on the contents of the tests directory in other packages. For example, when trying to build pipe_tasks against a Dockerized obs_base, I get:

 ____________________________________________ ReadDefectsTestCase.test_read_defects ____________________________________________ [gw3] linux -- Python 3.7.2 /opt/lsst/software/stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin/python3.7   self =     def setUp(self): > butler = dafPersist.ButlerFactory(mapper=BaseMapper()).create()   tests/test_read_CuratedCalibs.py:61:  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/test_read_CuratedCalibs.py:48: in __init__  policy = dafPersist.Policy(os.path.join(ROOT, "BaseMapper.yaml")) /opt/lsst/software/stack/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/19.0.0-1-g6fe20d0+1/python/lsst/daf/persistence/policy.py:80: in __init__  self.__initFromFile(other) /opt/lsst/software/stack/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/19.0.0-1-g6fe20d0+1/python/lsst/daf/persistence/policy.py:111: in __initFromFile  self.__initFromYamlFile(path) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _   self = {} path = '/opt/lsst/software/stack/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_base/19.0.0-9-ge91d8c4+1/tests/BaseMapper.yaml'    def __initFromYamlFile(self, path):  """Opens a file at a given path and attempts to load it in from yaml.    :param path:  :return:  """ > with open(path, 'r') as f: E FileNotFoundError: [Errno 2] No such file or directory: '/opt/lsst/software/stack/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_base/19.0.0-9-ge91d8c4+1/tests/BaseMapper.yaml'   /opt/lsst/software/stack/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/19.0.0-1-g6fe20d0+1/python/lsst/daf/persistence/policy.py:145: FileNotFoundError 

This is happening because $PIPE_TASKS_DIR/tests/test_read_CuratedCalibs.py depends upon$OBS_BASE_DIR/tests/BaseMapper.yaml, which has been removed from the Docker images.

This renders the Docker images much less useful for development than they might otherwise be.

I don't know what the original motivation for stripping tests was (just to save space?). In general, I'd suggest that the Docker images should contain exactly the contents of the packages published at eups.lsst.codes — if it's appropriate to strip something from the Docker image, it must be appropriate to strip it from the package, and vice versa. Please stop special-casing this directory in image construction.

Adding Josh and Simon as watchers here, as respectively the author of the Docker image building code and the tests that are being broken.

## People

• Assignee:
Frossie Economou
Reporter:
John Swinbank
Watchers:
John Swinbank, Joshua Hoblitt, Simon Krughoff, Tim Jenness