Meredith Rawls: I think you're probably in the best place to review this. I suspect it may change some of the measured metrics. Probably the best way to test it is to run through each dataset's pipeline and see if things are still happy?

Question: the gen3 butler doesn't allow `-` in a dataset name, so the `pan-starrs` refcat in these test data have to be renamed. I would probably recommend calling it either `ps1` (which is the data release it came from) or just `panstarrs` (even though the real name has a hyphen). Should I make that change on this ticket, or should we do it on another one? Would we need to change any configs other than the two in both ap_verify_hits `config/` directories?

Note that there are two PS1 data releases (DR1 and DR2).

A few things.

The changes in the datasets themselves look fine, and thanks (John) for explaining HTM trixels and "depth" to me via DM.

I ran ap_verify with the CI dataset on this ticket branch, and it failed with

apPipe.ccdProcessor.calibrate.astromRefObjLoader INFO: Loading reference objects using center (154.967854, -5.784114) and radius 0.19703074992397315 deg

apPipe FATAL: Failed on dataId={'date': '2015-02-18', 'filter': 'g', 'visit': 411420, 'hdu': 2, 'ccdnum': 5, 'object': 'Blind15A_40'}: RuntimeError: Unknown reference filter phot_g_mean_mag_flux 

I think this failure is related to the new refcat and needs to be addressed before this ticket is done.

It then subsequently failed with

AttributeError: lsst.pipe.base.config.Connections has no attribute taskName 

Which, incidentally, I am beginning to think is the new "no locations for get" (in its pervasiveness and opaqueness, not what it actually means). I believe this is the ap_pipe bug Krzysztof Findeisen caught and fixed earlier this week, so for now, don't worry about it.

Regarding Pan-STARRS as the photometric refcat. Please specify in both datasets' READMEs that it includes Gaia DR2 and Pan-STARRS PS1 DRX (I don't know if it's DR1 or DR2?), per Eric's comment above. I am quite annoyed that gen3 does not permit hyphens in refcat names, as this is the kind of thing that will create significant overhead for folks who are just trying to do a processCcd rerun and suddenly it doesn't work anymore. That said, assuming it truly is necessary to banish hyphens from our refcats forever, I think it is fine to update the `pan-starrs` name to be hyphen free on this ticket. I think I would vote for pan_starrs or panstarrs (and not, e.g., PS1).

Yes, I fixed it on DM-22641. (For the record, that error is not from the Butler at all: I was assigning to a config field that didn't exist. I think the AttributeError might have been raised by Python itself.)

Thank you, Meredith Rawls. I've made some changes and got a successful run of ci_hits2015 on my desktop (I'm running the full hits2015 test overnight now). Could you please try it again and see if it works for you?

Thanks! I also ran the CI dataset and made a quick plot of the good DIA Sources on the sky on this ticket branch vs. the current master (plot is attached to this ticket). It looks essentially the same; the new version has 337 good sources vs. 362 good sources in master. Assuming nothing falls over when you run the full hits2015, I think this is all set.

Thanks Meredith Rawls; the full run didn't produce errors, so we should be good.

Merged and done.