[DM-4181] Butler: add support for skymap based dataIds - Jira

Nate Pease [X] (Inactive) created issue - 23/Oct/15 3:56 PM

Frossie Economou made changes - 30/Oct/15 10:40 AM

Field	Original Value	New Value
Component/s		butler [ 12317 ]
Team		Data Access and Database [ 10204 ]

Nate Pease [X] (Inactive) made changes - 01/Dec/15 3:35 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 01/Dec/15 3:35 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 01/Dec/15 6:21 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

John Swinbank made changes - 02/Dec/15 11:39 AM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

John Swinbank made changes - 02/Dec/15 11:43 AM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Hide

Permalink

Jim Bosch added a comment - 02/Dec/15 1:00 PM

I don't think we can get away with hard-coding the skymap configuration in the mapper policy file, at least as those policy files stand now; each camera needs to be able to support multiple skymap configurations (though not within the same data repository).

Show

Jim Bosch added a comment - 02/Dec/15 1:00 PM I don't think we can get away with hard-coding the skymap configuration in the mapper policy file, at least as those policy files stand now; each camera needs to be able to support multiple skymap configurations (though not within the same data repository).

Jim Bosch made changes - 02/Dec/15 1:31 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Jim Bosch made changes - 02/Dec/15 1:36 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 2:33 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:00 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:01 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:19 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:26 PM

Description

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{{datasetType = 'coaddTempExp'
key = (?)
format = ['patch', 'visit']
dataId = {'tract':<some numberOrId>}
myButler.queryMetadata(datasetType, key, format, dataId)}}

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:27 PM

Description

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{{datasetType = 'coaddTempExp'
key = (?)
format = ['patch', 'visit']
dataId = {'tract':<some numberOrId>}
myButler.queryMetadata(datasetType, key, format, dataId)}}

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{{datasetType = 'coaddTempExp'
key = (?)
format = ['patch', 'visit']
dataId = {'tract':<some numberOrId>}
myButler.queryMetadata(datasetType, key, format, dataId)
}}

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:27 PM

Description

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{{datasetType = 'coaddTempExp'
key = (?)
format = ['patch', 'visit']
dataId = {'tract':<some numberOrId>}
myButler.queryMetadata(datasetType, key, format, dataId)
}}

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{{
datasetType = 'coaddTempExp'
key = (?)
format = ['patch', 'visit']
dataId = {'tract':<some numberOrId>}
myButler.queryMetadata(datasetType, key, format, dataId)
}}

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:40 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:42 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 3:55 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 4:08 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 4:09 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 4:48 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 5:14 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 5:17 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 5:30 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 5:32 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 02/Dec/15 5:33 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Nate Pease [X] (Inactive) made changes - 03/Dec/15 1:36 PM

Link

This issue relates to ~~DM-4532~~ [ ~~DM-4532~~ ]

Robert Lupton made changes - 03/Dec/15 4:58 PM

Remote Link

This issue links to "Page (Confluence)" [ 13228 ]

Jacek Becla made changes - 07/Dec/15 11:39 AM

Rank

Ranked higher

Jacek Becla made changes - 08/Dec/15 3:26 PM

Remote Link

This issue links to "Page (Confluence)" [ 13256 ]

Nate Pease [X] (Inactive) made changes - 08/Dec/15 3:32 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Tim Jenness made changes - 08/Dec/15 4:24 PM

Remote Link

This issue links to "Page (Confluence)" [ 13256 ]

Kian-Tat Lim made changes - 08/Dec/15 4:28 PM

Remote Link

This issue links to "Page (Confluence)" [ 13256 ]

Kian-Tat Lim made changes - 08/Dec/15 4:30 PM

Remote Link

This issue links to "Page (Confluence)" [ 13256 ]

Kian-Tat Lim made changes - 08/Dec/15 4:40 PM

Remote Link

This issue links to "Page (Confluence)" [ 13256 ]

Xiuqin Wu [X] (Inactive) made changes - 09/Dec/15 8:07 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Jacek Becla made changes - 16/Dec/15 11:40 AM

Description

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{{
datasetType = 'coaddTempExp'
key = (?)
format = ['patch', 'visit']
dataId = {'tract':<some numberOrId>}
myButler.queryMetadata(datasetType, key, format, dataId)
}}

load the skymap from the repository and use it a la metadata lookup to complete lookups. for example if the script wants all the patches that are in tract x, look in the skymap to get that information.

will be something like:
create a skymap object given a configuration (in the repository) (it has its own dataset type)
create a registry with that skymap object
use that registry to lookup skymap related parameters. Might need to add to the policy file that a given configuration uses the skymap.

requires that data is already indexable by tract & patch, the work is adds iteration over tract/patch specified by the skymap.

example use case:
{code}
{
    datasetType = 'coaddTempExp'
    key = (?)
    format = ['patch', 'visit']
    dataId = {'tract':<some numberOrId>}
    myButler.queryMetadata(datasetType, key, format, dataId)
}
{code}

Jacek Becla made changes - 16/Dec/15 11:41 AM

Link

This issue relates to ~~DM-3472~~ [ ~~DM-3472~~ ]

Nate Pease [X] (Inactive) made changes - 16/Dec/15 11:52 AM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 16/Dec/15 11:56 AM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 16/Dec/15 11:58 AM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Jacek Becla made changes - 16/Dec/15 12:09 PM

Epic Link

DM-4341 [ 21422 ]

Jacek Becla made changes - 16/Dec/15 12:14 PM

Story Points

15

Jacek Becla made changes - 14/Jan/16 2:49 PM

Rank

Ranked higher

Jacek Becla made changes - 14/Jan/16 2:49 PM

Rank

Ranked higher

Jacek Becla made changes - 14/Jan/16 2:49 PM

Rank

Ranked higher

Jacek Becla made changes - 14/Jan/16 2:50 PM

Rank

Ranked higher

Nate Pease [X] (Inactive) made changes - 20/Jan/16 3:13 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 20/Jan/16 3:15 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 20/Jan/16 3:17 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 20/Jan/16 3:17 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 20/Jan/16 3:18 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Nate Pease [X] (Inactive) made changes - 20/Jan/16 3:57 PM

Rank

Ranked higher

Nate Pease [X] (Inactive) made changes - 02/Feb/16 5:20 PM

Rank

Ranked higher

Nate Pease [X] (Inactive) made changes - 07/Mar/16 1:59 PM

Remote Link

This issue links to "Page (Confluence)" [ 13260 ]

Fritz Mueller made changes - 16/Mar/16 7:54 AM

Rank

Ranked lower

Nate Pease [X] (Inactive) made changes - 09/May/16 3:05 PM

Epic Link

DM-4341 [ 21422 ]

DM-6047 [ 24358 ]

Fritz Mueller made changes - 24/May/16 7:38 AM

Epic Link

DM-6047 [ 24358 ]

DM-4341 [ 21422 ]

Fritz Mueller made changes - 24/May/16 7:38 AM

Epic Link

DM-4341 [ 21422 ]

DM-6047 [ 24358 ]

Fritz Mueller made changes - 02/Jun/16 12:05 PM

Epic Link

DM-6047 [ 24358 ]

~~DM-4340~~ [ 21421 ]

Fritz Mueller made changes - 05/Dec/16 3:44 PM

Sprint

DB_S17_12 [ 304 ]

Fritz Mueller made changes - 05/Dec/16 6:35 PM

Sprint

DB_S17_12 [ 304 ]

DB_S17_01 [ 307 ]

Fritz Mueller made changes - 05/Dec/16 6:35 PM

Rank

Ranked higher

Fritz Mueller made changes - 05/Dec/16 6:43 PM

Rank

Ranked lower

Fritz Mueller made changes - 05/Dec/16 6:44 PM

Sprint

DB_S17_01 [ 307 ]

Fritz Mueller made changes - 05/Dec/16 6:44 PM

Rank

Ranked higher

Fritz Mueller made changes - 30/May/17 9:50 AM

Epic Link

~~DM-4340~~ [ 21421 ]

~~DM-10678~~ [ 32630 ]

Fritz Mueller made changes - 06/Jun/17 3:26 PM

Rank

Ranked higher

Fritz Mueller made changes - 05/Oct/17 2:48 PM

Rank

Ranked higher

Fritz Mueller made changes - 20/Nov/17 1:20 AM

Epic Link

~~DM-10678~~ [ 32630 ]

~~DM-12759~~ [ 36358 ]

Tim Jenness made changes - 02/Aug/21 10:19 PM

Remote Link

This issue links to "Page (Confluence)" [ 29416 ]

Hide

Permalink

Tim Jenness added a comment - 02/Aug/21 10:36 PM

Gen 2 is deprecated and Gen3 handles skymaps in its own way.

Show

Tim Jenness added a comment - 02/Aug/21 10:36 PM Gen 2 is deprecated and Gen3 handles skymaps in its own way.

Tim Jenness made changes - 02/Aug/21 10:36 PM

Resolution		Done [ 10000 ]
Status	To Do [ 10001 ]	Won't Fix [ 10405 ]

Data Management

Butler: add support for skymap based dataIds

Details

Description

Attachments

Issue Links

Activity

People

Dates

Jenkins