Load data from one table to another every 10 mins - Cassandra -


we have stream of data coming table every 10 mins. no history preserved. existing data has flushed new table b every time data loaded in table a. can done dynamically or automated in cassandra?

i can think of loading table csv file , loading table b every time table flushed. have done @ database level itself. ideas or suggestions appreciated.

thanks, arun

for smaller amounts of data put cron:

https://dba.stackexchange.com/questions/58901/what-is-a-good-way-to-copy-data-from-one-cassandra-columnfamily-to-another-on-th

if larger , running newer versions of cassandra (3.8+)

http://cassandra.apache.org/doc/latest/operating/cdc.html https://issues.apache.org/jira/browse/cassandra-8844

and replay data table need (by sort of outside process, script, app etc ...).

basically there tools around like: https://github.com/carloscm/cassandra-commitlog-extract

you use samples there cover use-case.

but use cases handled @ application level, writes relatively cheap cassandra.


Comments

Popular posts from this blog

php - Permission denied. Laravel linux server -

google bigquery - Delta between query execution time and Java query call to finish -

python - Pandas two dataframes multiplication? -