Okay, Avro serialization and the schema are now pulled out of alert-stream-simulator and its streamsim package. It now references those things from lsst.alert.packet, which is marked as a dependency, and which it can get from PyPI.
The streamsim package still has more in it that we could take out, but this other stuff is less obviously redundant. Specifically:
- streamsim.serialization is now mostly to do with serializing timestamps (in a way that really only matters internally to alert-stream-simulator) and with serializing to Confluent Wire Format (which only matters when you write to Kafka). The timestamp stuff should definitely stay in alert-stream-simulator; the wire format stuff could conceivably belong in some new lsst.alert.stream.kafka package.
- streamsim._kafka is a large pile of stuff for interacting with Kafka sanely. That could all go elsewhere, although the API was written with the simulator in mind.
Personally, I think code reuse can be overrated, and we shouldn't fret too much about throwing this stuff out and doing it over when we do the actual Kafka client used in the production pipeline. That thing will have different needs anyway. So, I lean towards calling this good.
But I want to raise the question - should we pull more out of alert-stream-simulator and its streamsim package, or are we good?