Sooo... this issue is at least moderately subtle. Further, I'm hardly an expert on image differencing algorithms so Gabor Kovacs (or whoever) will have to correct my misapprehensions.
However. What's at issue here is: how do you generate a list of sources to use to generate your differencing kernel?
There are broadly two options (at least, in the code; there may be other ways of doing it in principle).
- You can get a list of appropriate sources in the field of view from your reference catalog; or
- You can get source lists from the science and template images, then cross-match them to get a list of sources that appear in both.
My hunch is that the second is likely to work better, since that's relying on what's actually in the data rather than what you hope is in the data. However, this is the sort of thing that Gabor Kovacs (or Eric Bellm) might have more educated opinions on than I do.
However, it looks like the author of this code agreed with my hunch, since that's what it attempts to do by default. And, of course, in order to do that it wants a list of sources from the science image (which it can get either from the Butler, or by running source detection) and from the template, which it assumes it can get from the Butler.
Now, you are thinking that's a questionable assumption. But, in fact, in operations it's not: templates are generated during data release production, which has already done exquisite source detection and stashed those catalogues away for later use. Relying on them rather than doing your own thing is the right thing to do.
So what does that mean? Well, there are a couple of options here:
- When we are ingesting templates, we could/should also be ingesting source lists to go with them. That would be more realistic, in terms of being similar to how things will work in operations.
- We could change the default setting to rely on fetching sources from the reference catalog, rather than from actual detections. We'd have to rely on Gabor & Eric to advise on whether they regard that as scientifically wise.
- We could just run a sourcefinding step on the template as part of the AP pipeline, which you don't like.
In any event, I don't think this one is easy to “fix” without a deeper understanding of what ought to be happening.