# Attempt to re-register the LATISS instrument with butler register-instrument fails for /repo/main

XMLWordPrintable

#### Details

• Type: Story
• Status: To Do
• Resolution: Unresolved
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
Data Facility
• Urgent?:
No

#### Description

Newer versions of the LSST Stack introduced new filters for LATISS instrument. As a result, ingest attempts for Auxtel images to the /repo/main at NCSA fail due to the Butler not recognizing them. For example:

 psycopg2.errors.ForeignKeyViolation: insert or update on table "exposure" violates foreign key constraint "fkey_exposure_physical_filter_instrument_name_instrume_ee4bb371" DETAIL: Key (instrument, physical_filter)=(LATISS, FELH0600~empty) is not present in table "physical_filter". 

According to the discussion in this thread, the new version of the instrument (introducing the new filters) can be re-registered with butler register-instrument. However, my attempt to do that failed with the following error:

 lsst.daf.butler.registry.interfaces._database.DatabaseConflictError: Conflict in sync for table instrument on column(s) visit_max: 3050123199999 != 6050123199999, exposure_max: 3050123199999 != 6050123199999. 

The full traceback can be found here.

#### Activity

Hide
Tim Jenness added a comment -

Yes, that change happened in DM-16297 I think. There would need to be a way to pass down a flag from register-instrument saying that items that are different can be overwritten.

Show
Tim Jenness added a comment - Yes, that change happened in DM-16297 I think. There would need to be a way to pass down a flag from register-instrument saying that items that are different can be overwritten.
Hide
Jim Bosch added a comment -

That doesn't make sense to me; DM-16297 was merged long before this repo was created (which was February of this year).  My previous guess from some git blaming (not rigorously confirmed) was

and there's more discussion on this slack thread:

https://lsstc.slack.com/archives/C2JPT1KB7/p1631885321252900

I can go make the code changes to add a update kwarg/option to instrument registration, but I'd really like somebody else to check whether the new or old values are actually the desired ones for LATISS.

Show
Jim Bosch added a comment - That doesn't make sense to me; DM-16297 was merged long before this repo was created (which was February of this year).  My previous guess from some git blaming (not rigorously confirmed) was https://github.com/lsst/obs_lsst/commit/16a9178d8f4d6119ad9d21a6940fa5c86cdb16a1 and there's more discussion on this slack thread: https://lsstc.slack.com/archives/C2JPT1KB7/p1631885321252900 I can go make the code changes to add a update kwarg/option to instrument registration, but I'd really like somebody else to check whether the new or old values are actually the desired ones for LATISS.
Hide
Tim Jenness added a comment -

Jim Bosch neither of us are right. My change moved from hard-coded to using the general code. The change you mention optimized the calculation for arbitrary controllers. I think the real problem was caused by DM-30142 which added new controllers to the list and that forced the number to go up.

This code assumes that when visit != exposure the visit we do generate is a smaller int than the one coming from exposure. I'd have to check on that since I can't remember.

Show
Tim Jenness added a comment - Jim Bosch neither of us are right. My change moved from hard-coded to using the general code. The change you mention optimized the calculation for arbitrary controllers. I think the real problem was caused by DM-30142 which added new controllers to the list and that forced the number to go up. This code assumes that when visit != exposure the visit we do generate is a smaller int than the one coming from exposure. I'd have to check on that since I can't remember.
Hide
Jim Bosch added a comment -

Ok, "new controller" makes it sounds like the new values are indeed better, so a forced update is the way to go.

DM-31903 has the functionality for forced updates and is now in review, so this ticket can just be re-running butler register-instrument with that branch.

Show
Jim Bosch added a comment - Ok, "new controller" makes it sounds like the new values are indeed better, so a forced update is the way to go. DM-31903 has the functionality for forced updates and is now in review, so this ticket can just be re-running butler register-instrument with that branch.

#### People

Assignee:
Unassigned
Reporter:
Mikolaj Kowalik
Watchers:
Jim Bosch, Mikolaj Kowalik, Tim Jenness