# Convert stack demo refcat to HTM indexed

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
3
• Team:
SQuaRE

## Description

lsst_dm_stack_demo uses an astrometry.net reference catalog. In order to remove a.net support from the stack, we need to convert this refcat from a.net format to HTM Indexed format. Simon Krughoff claimed he had some code to do such conversions in the past.

Converting the existing refcat is probably better than creating a new refcat because any changes to the refcat contents would be reflected in changes in the demo output, and we don't want to have to go and vet those changes again.

## Activity

Hide
Simon Krughoff added a comment -

This is done, as far as I'm concerned. Note that this also turned up a bug in the FITS reader.

Show
Simon Krughoff added a comment - This is done, as far as I'm concerned. Note that this also turned up a bug in the FITS reader.
Hide
Simon Krughoff added a comment - - edited

Please have a look at the two PRs associated with this ticket. Note that there was a bug in the FITS reader that I fixed in meas_algorithms using astropy Table s.

Show
Simon Krughoff added a comment - - edited Please have a look at the two PRs associated with this ticket. Note that there was a bug in the FITS reader that I fixed in meas_algorithms using astropy Table s.
Hide
Simon Krughoff added a comment -

Two more things:
1) build for this ticket is here
2) This is considered a blocker for the v19 release, so if it's not something you can review relatively quickly, we should re-assign it.

Show
Simon Krughoff added a comment - Two more things: 1) build for this ticket is here 2) This is considered a blocker for the v19 release, so if it's not something you can review relatively quickly, we should re-assign it.
Hide
Simon Krughoff added a comment -

I was asked for some timing.

 >>> def from_fits(): ... hdl = fits.open('stars.fits') ... arr = hdl[1].data ... >>> def from_table(): ... table = Table.read('stars.fits') ... arr = table.as_array() ... >>> timeit(from_table, number=500) 7.41295113787055 >>> timeit(from_fits, number=500) 4.18857777165249 >>> timeit(from_table, number=500) 7.639025655109435 >>> timeit(from_fits, number=500) 3.9214696479029953 

So, going straight to array from FITS is significantly faster, but I don't think it's a deal breaker. The Table version does give us the big benefit of putting the values in native-endian order out of the box. Otherwise we'd have to play games with the arrays read in via fits.open.

Show
Simon Krughoff added a comment - I was asked for some timing. >>> def from_fits(): ... hdl = fits.open('stars.fits') ... arr = hdl[1].data ... >>> def from_table(): ... table = Table.read('stars.fits') ... arr = table.as_array() ... >>> timeit(from_table, number=500) 7.41295113787055 >>> timeit(from_fits, number=500) 4.18857777165249 >>> timeit(from_table, number=500) 7.639025655109435 >>> timeit(from_fits, number=500) 3.9214696479029953 So, going straight to array from FITS is significantly faster, but I don't think it's a deal breaker. The Table version does give us the big benefit of putting the values in native-endian order out of the box. Otherwise we'd have to play games with the arrays read in via fits.open .
Hide
Eli Rykoff added a comment -

Thanks. This is good to know for further reference.

Show
Eli Rykoff added a comment - Thanks. This is good to know for further reference.

## People

• Assignee:
Simon Krughoff
Reporter:
John Parejko
Reviewers:
Eli Rykoff
Watchers:
Eli Rykoff, Gabriele Comoretto, John Parejko, John Swinbank, Paul Price, Robert Lupton, Simon Krughoff, Tim Jenness