Paul Price added a comment - 21/Mar/17 11:37 AM DM-9438 done: price@pap-laptop:~/hsc/obs_subaru (master=) $ git rebase --onto master e98f607 lsst/master First, rewinding head to replay your work on top of it... Applying: config: use PS1 reference catalog in LSST format price@pap-laptop:~/hsc/obs_subaru ((ddceaac6...)) $ git co master Warning: you are leaving 1 commit behind, not connected to any of your branches:   ddceaac6 config: use PS1 reference catalog in LSST format   If you want to keep it by creating a new branch, this may be a good time to do so with:   git branch <new-branch-name> ddceaac6   Switched to branch 'master' Your branch is up-to-date with 'origin/master'. price@pap-laptop:~/hsc/obs_subaru (master=) $ git merge ddceaac6 Updating e09d034b..ddceaac6 Fast-forward config/Mosaic.py | 13 ++++--------- config/measureCoaddSources.py | 4 +++- config/processCcd.py | 14 ++++---------- 3 files changed, 11 insertions(+), 20 deletions(-) price@pap-laptop:~/hsc/obs_subaru (master>) $ git out commit ddceaac6341899e2a1ee6c30cc02ca031617e08d Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 13:55:30 2017 -0400   config: use PS1 reference catalog in LSST format This frees us from dependency on astrometry_net, and removes our reliance on eups for setting the configuration. price@pap-laptop:~/hsc/obs_subaru (master>) $ git push Counting objects: 6, done. Delta compression using up to 8 threads. Compressing objects: 100% (6/6), done. Writing objects: 100% (6/6), 961 bytes | 0 bytes/s, done. Total 6 (delta 5), reused 0 (delta 0) remote: Resolving deltas: 100% (5/5), completed with 4 local objects. To github.com:HyperSuprime-Cam/obs_subaru.git e09d034b..ddceaac6 master -> master

Paul Price added a comment - 21/Mar/17 5:31 PM DM-9810 and DM-9856 done: price@pap-laptop:~/hsc/meas_extensions_psfex (master=) $ git rebase -p --onto master f76c79a lsst/master Successfully rebased and updated detached HEAD. price@pap-laptop:~/hsc/meas_extensions_psfex ((423c6dd...)) $ git co master Warning: you are leaving 2 commits behind, not connected to any of your branches:   423c6dd Merge branch 'tickets/DM-9810' 9b2b692 Add config parameter for BASIS_TYPE   If you want to keep them by creating a new branch, this may be a good time to do so with:   git branch <new-branch-name> 423c6dd   Switched to branch 'master' Your branch is up-to-date with 'origin/master'. price@pap-laptop:~/hsc/meas_extensions_psfex (master=) $ git merge 423c6dd Updating 1b4ab0c..423c6dd Fast-forward python/lsst/meas/extensions/psfex/psfexPsfDeterminer.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) price@pap-laptop:~/hsc/meas_extensions_psfex (master>) $ git out commit 423c6dd4409b74f58f625e99ccea30baf5033800 Merge: 1b4ab0c 9b2b692 Author: Bob Armstrong <rearmstr@gmail.com> Date: Tue Mar 21 15:38:24 2017 -0400   Merge branch 'tickets/DM-9810'   commit 9b2b69271208aaddd3a1aaeb77066c6fac4fecdc Author: Bob Armstrong <rearmstr@gmail.com> Date: Fri Mar 17 13:05:18 2017 -0400   Add config parameter for BASIS_TYPE price@pap-laptop:~/hsc/meas_extensions_psfex (master>) $ git push Counting objects: 9, done. Delta compression using up to 8 threads. Compressing objects: 100% (8/8), done. Writing objects: 100% (9/9), 1.05 KiB | 0 bytes/s, done. Total 9 (delta 4), reused 0 (delta 0) remote: Resolving deltas: 100% (4/4), completed with 3 local objects. To github.com:HyperSuprime-Cam/meas_extensions_psfex.git 1b4ab0c..423c6dd master -> master       price@pap-laptop:~/hsc/obs_subaru (master=) $ git rebase -p --onto master 60c4f2d lsst/master Successfully rebased and updated detached HEAD. price@pap-laptop:~/hsc/obs_subaru ((3ec4d6f1...)) $ git co master Warning: you are leaving 2 commits behind, not connected to any of your branches:   3ec4d6f1 Merge branch 'tickets/DM-9810' ee7cef3f Change psfex oversampling options.   If you want to keep them by creating a new branch, this may be a good time to do so with:   git branch <new-branch-name> 3ec4d6f1   Switched to branch 'master' Your branch is up-to-date with 'origin/master'. price@pap-laptop:~/hsc/obs_subaru (master=) $ git merge 3ec4d6f1 Updating ddceaac6..3ec4d6f1 Fast-forward config/processCcd.py | 3 +++ 1 file changed, 3 insertions(+) price@pap-laptop:~/hsc/obs_subaru (master>) $ git out commit 3ec4d6f12002a8396fa636398d5920e0bf56511e Merge: ddceaac6 ee7cef3f Author: Bob Armstrong <rearmstr@gmail.com> Date: Tue Mar 21 15:42:08 2017 -0400   Merge branch 'tickets/DM-9810'   commit ee7cef3ff06583f094a897451d9e6f157f1bc164 Author: Bob Armstrong <rearmstr@gmail.com> Date: Tue Mar 21 12:42:32 2017 -0400   Change psfex oversampling options. This commit forces psfex to use the native pixel basis if the FWHM > 3 pixels. Otherwise, it will 2x oversample the PSF model. price@pap-laptop:~/hsc/obs_subaru (master>) $ git push Counting objects: 5, done. Delta compression using up to 8 threads. Compressing objects: 100% (5/5), done. Writing objects: 100% (5/5), 661 bytes | 0 bytes/s, done. Total 5 (delta 4), reused 0 (delta 0) remote: Resolving deltas: 100% (4/4), completed with 3 local objects. To github.com:HyperSuprime-Cam/obs_subaru.git ddceaac6..3ec4d6f1 master -> master       price@pap-laptop:~/hsc/meas_deblender (master=) $ git rebase -p --onto master e6413fc lsst/master Successfully rebased and updated detached HEAD. price@pap-laptop:~/hsc/meas_deblender ((dadfd13...)) $ git co master Warning: you are leaving 2 commits behind, not connected to any of your branches:   dadfd13 Merge branch 'tickets/DM-9856' 042d331 Add medianSmoothTemplate as config parameter   If you want to keep them by creating a new branch, this may be a good time to do so with:   git branch <new-branch-name> dadfd13   Switched to branch 'master' Your branch is up-to-date with 'origin/master'. price@pap-laptop:~/hsc/meas_deblender (master=) $ git merge dadfd13 Updating fd75620..dadfd13 Fast-forward python/lsst/meas/deblender/deblend.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) price@pap-laptop:~/hsc/meas_deblender (master>) $ git out commit dadfd1301d4505e3d505e6cf0a5034402a9dfabb Merge: fd75620 042d331 Author: Bob Armstrong <rearmstr@gmail.com> Date: Tue Mar 21 15:37:14 2017 -0400   Merge branch 'tickets/DM-9856'   commit 042d33132eaabeb9e597c994a4af21396c79983c Author: Bob Armstrong <rearmstr@gmail.com> Date: Fri Mar 17 13:06:15 2017 -0400   Add medianSmoothTemplate as config parameter price@pap-laptop:~/hsc/meas_deblender (master>) $ git push Counting objects: 8, done. Delta compression using up to 8 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (8/8), 836 bytes | 0 bytes/s, done. Total 8 (delta 4), reused 0 (delta 0) remote: Resolving deltas: 100% (4/4), completed with 3 local objects. To github.com:HyperSuprime-Cam/meas_deblender.git fd75620..dadfd13 master -> master

Paul Price added a comment - 21/Mar/17 7:13 PM DM-9812 done: price@pap-laptop:~/hsc/obs_base (master=) $ git rebase -p --onto master 7f06ade lsst/master   price@pap-laptop:~/hsc/meas_astrom (master=) $ git rebase -p --onto master e3a4b3c lsst/master ==> Conflict in __init__.py easily resolved ==> Added commit to adapt use of SchemaMapper.join   price@pap-laptop:~/hsc/pipe_tasks (master=) $ git rebase -p --onto master 4569002 lsst/master   price@pap-laptop:~/hsc/obs_subaru (master=) $ git rebase -p --onto master 7d65835 lsst/master

Paul Price added a comment - 22/Mar/17 9:38 AM Released: This new beta release incorporates some features and bugfixes we anticipate needing for the production run: • DM-9541 : Bug related to MPI pickling when running coaddDriver • DM-9812 : Clean up outputs from CharacterizeImageTask and CalibrateTask • DM-9438 : Switch default reference catalog for HSC to PS1 in LSST format • DM-9810 : Make PSFEx oversampling configurable • DM-9856 : Add option to deblender to turn off smoothing. • DM-9871 : Move wcs and fcr datasets out of pointing directory in obs_subaru • DM-9874 : Increase matcher.numBrightStars This should allow some basic verification runs to start so we can iron out any obvious bugs. Installation instructions are appended below. Cheers, Paul. Installation notes: 1. Do not install the new hscPipe into the same eups stack as previous versions; prepare a new directory that will hold this and subsequent releases of hscPipe. 2. If you have installed the LSST pipeline, you can try installing into that eups stack (e.g., EUPS_PKGROOT= http://master.ipmu.jp/~price/packages-hscPipe5 eups distrib install hscPipe 5.0-beta1). I haven’t tested that mode yet, but I hope that it will work and not corrupt anything... 3. The software now uses the system version of various third-party packages (e.g., python, numpy, scipy) if they are recent enough. You will be given an option to install Miniconda Python (a miniature version of Anaconda Python) that contains the required packages. We recommend that you do choose to install Miniconda, even if you are confident that you have recent packages, because it will simplify the process in the event you have any problems or notice any bugs. 4. The HSC pipeline now has support for python 3, but I've not yet forked the miniconda3 package. You can try using your own python 3 packages if you want... 5. We now support building the HSC pipeline on Mac OSX with the System Integrity Protection (SIP) feature. 6. Make sure you have the necessary pre-requisites installed. In particular, you need a recent C++ compiler that can handle C++11 (e.g., gcc >= 4.8). See the LSST install page ( https://pipelines.lsst.io/install/newinstall.html#prerequisites ) for details and installation instructions. Installation instructions: -1. Install the pre-requisites. See https://pipelines.lsst.io/install/newinstall.html#prerequisites . 0. Start off with a fresh shell. 1. Set up the directory that will host the installation. mkdir /path/to/hscPipe5 cd /path/to/hscPipe5 2. Set the environment: drop any old stack environment, point to the new stack package source (which will likely change for future releases). unset EUPS_DIR EUPS_SHELL EUPS_PATH EUPS_PKGROOT SETUP_EUPS export EUPS_PKGROOT= http://master.ipmu.jp/~price/packages-hscPipe5/ 3. Ensure you're using a suitable C++ compiler: need gcc >= 4.8 on Linux. If you don't have it in your environment, take the necessary steps to load it. 4. Download the install script: wget $EUPS_PKGROOT/newinstall.sh 5. Run the install script; we recommend entering "yes" when asked if you want to install Miniconda. bash newinstall.sh 6. Source the appropriate setup script for your shell. You'll want to do this each time you use the HSC stack; you may want to put it in your shell startup script. source /path/to/hscPipe5/loadLSST.bash 7. Install the rest of the stack. For future releases, you'll be able to just run this. A rather long pause (5-10 minutes?) after installing the first package (apr) appears to be common — be patient. eups distrib install hscPipe 5.0-beta1 Support: 1. Look at the LSST installation pages: https://pipelines.lsst.io/install/newinstall.html 2. Search the LSST forum for reports of similar problems: https://community.lsst.org 3. Ask a question on the LSST forum 4. Ask a question on this mailing list