Data Lake

Sravz stores data in varaious data sources and seamlessly manages the data. The data is processed by the task runners and displayed on the dashboard

Data Lake Resources

• SQL Databases (Mysql/Postgres)
• NoSQL Databases (MongoDB/Redis)
• Data Stores (S3/Cloud Storage/Blob Store/Ceph)

Screenshots of Data Lake Components

• MongoDB

• Redis

• AWS S3

  $ aws s3 ls s3://XXXX/ | head
    2022-07-31 20:00:19       5952 bond_us_ch0200252788.json
    2022-07-31 20:00:20       3968 bond_us_fr0012938116.json
    2022-07-31 20:00:21      28625 bond_us_nl0000113587.json
    2022-07-31 20:00:22      14932 bond_us_nl0000116168.json
    2022-07-31 20:00:23       4861 bond_us_nl0010733424.json
    2022-07-31 20:00:44       4355 bond_us_us00165cap95.json
    2022-07-31 20:00:45       2952 bond_us_us00175pac77.json
    2022-07-31 20:00:47       4013 bond_us_us001846aa20.json
    2022-07-31 20:01:46       3801 bond_us_us003000aa44.json
    2022-07-31 20:01:47       3903 bond_us_us00404aap49.json