Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Story Points:2
-
Epic Link:
-
Sprint:TSSW Sprint - Apr 11 - Apr 25
-
Team:Telescope and Site
-
Urgent?:No
Description
It turns out we can easily support schema evolution in Kafka/Avro by specifying a default value for each field in the schema created by the SAL/Kafka producers.
This is a simple change. The main question is whether the default float value should be 0 (like DDS) or NaN. I doubt it makes much difference, because by the time the SAL/Kafka producer sees the SAL data, all fields that are in the current schema have been filled in. Thus this default only affects fields that we remove from the XML. I am inclined to use 0 because it appears NaN support in Avro is somewhat iffy and it matches OpenSplice.
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Story Points | 1 | 2 |
| Epic Link |
|
|
| Sprint | TSSW Sprint - Apr 11 - Apr 25 [ 1160 ] |
| Story Points | 2 | 1 |
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Story Points | 1 | 2 |
| Reviewers | Angelo Fausti [ afausti ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
| Description |
It turns out we can easily support schema evolution in Kafka/Avro by specifying a default value for each field in the schema created by the SAL/Kafka producers.
This is a simple change. The main question is whether the default float value should be 0 (like DDS) or NaN. I doubt it makes much difference, because by the time the SAL/Kafka producer sees the SAL data, all fields that are in the current schema have been filled in. Thus this default only affects fields that we remove from the XML. I am inclined to use NaN for the default float, since it is a the more principled choice. |
It turns out we can easily support schema evolution in Kafka/Avro by specifying a default value for each field in the schema created by the SAL/Kafka producers.
This is a simple change. The main question is whether the default float value should be 0 (like DDS) or NaN. I doubt it makes much difference, because by the time the SAL/Kafka producer sees the SAL data, all fields that are in the current schema have been filled in. Thus this default only affects fields that we remove from the XML. I am inclined to use 0 because it appears NaN isn't practical for long-term Kafka use. |
| Description |
It turns out we can easily support schema evolution in Kafka/Avro by specifying a default value for each field in the schema created by the SAL/Kafka producers.
This is a simple change. The main question is whether the default float value should be 0 (like DDS) or NaN. I doubt it makes much difference, because by the time the SAL/Kafka producer sees the SAL data, all fields that are in the current schema have been filled in. Thus this default only affects fields that we remove from the XML. I am inclined to use 0 because it appears NaN isn't practical for long-term Kafka use. |
It turns out we can easily support schema evolution in Kafka/Avro by specifying a default value for each field in the schema created by the SAL/Kafka producers.
This is a simple change. The main question is whether the default float value should be 0 (like DDS) or NaN. I doubt it makes much difference, because by the time the SAL/Kafka producer sees the SAL data, all fields that are in the current schema have been filled in. Thus this default only affects fields that we remove from the XML. I am inclined to use 0 because it appears NaN support in Avro is somewhat iffy and it matches OpenSplice. |