Why Do Anything?
The platforms that were in use, Exadata and Cloudera, failed to provide the needed services. The organization’s data had to be accurate and up-to-date for external users (law enforcement), while providing the ability of fast query retrieval using REST API.
The telecom needed to come up with an alternative solution that could handle the rising number of IPDRs, which were hundreds of times greater than CDRs in their peta-scale datasets.
They required fast integration and interface to their warrant query interface, the ability to scale up if needed, and fast adoption into the organization (with no additional training). Lastly, the 3 nines availability and strict security standards were mandatory, in addition to compliance with EU GDPR regulations.
- Near-real-time feedback
- Rapid ingestion
- Ability to rapidly join many tables with large datasets
- Ability to directly query the data lake without ingesting the data into the system
- ANSI SQL compatibility – easy to handle by teams
- Less allocated servers and storage
- 99.9% availability