Load data from one table to another every 10 mins - Cassandra -
we have stream of data coming table every 10 mins. no history preserved. existing data has flushed new table b every time data loaded in table a. can done dynamically or automated in cassandra?
i can think of loading table csv file , loading table b every time table flushed. have done @ database level itself. ideas or suggestions appreciated.
thanks, arun
for smaller amounts of data put cron:
if larger , running newer versions of cassandra (3.8+)
http://cassandra.apache.org/doc/latest/operating/cdc.html https://issues.apache.org/jira/browse/cassandra-8844
and replay data table need (by sort of outside process, script, app etc ...).
basically there tools around like: https://github.com/carloscm/cassandra-commitlog-extract
you use samples there cover use-case.
but use cases handled @ application level, writes relatively cheap cassandra.
Comments
Post a Comment