For some time I have been concerned about a potential race condition in the way Kafka ts_salobj gets historical data. The issue is what happens if a Kafka Consumer is computing topic offsets to read historical data while the Kafka broker is purging data? It is clear that the Consumer may specify offsets to data that no longer exists. The question is: what happens if a Consumer specifies a topic offset for which no data is available?

I did some prototype testing and research and found the following:

The default behavior is to ignore all historical data and only read new data, which is not at all what we want. However, Consumer option "auto.offset.reset": "earliest" will make the consumer read the oldest data, which is exactly what we want.

So: add that option when constructing the consumer, and also document each option so readers understand why it is present.