SQreamDB 2023 update!

Extended query engines connectivity and improved performance:

We are thrilled to announce the first SQreamDB release for 2023!

With each new release, we aim to allow SQream customers to Ask Bigger questions and process and query myriad amounts of data.

This release includes enhanced performance, improved security, and advanced connectivity features.

We introduce a new version release system that follows the more commonly used Major-Minor versioning schema, starting at the 4.0 version. 

Noteworthy features: 

• JSON data format support – SQream now natively supports reading from and writing to JSON file format. JSON (JavaScript Object Notation) is an open standard file format that enables the collection and transmission of semi-structured data in a text-based form. By embracing this widely-used format, SQreamDB automatically infers the schema of a JSON foreign table.
• Enhanced Security – We’ve enhanced security by adding LDAP (Lightweight Directory Access Protocol) support. This means that SQream customers who manage their users centrally through Active Directory can use their company’s authentication credentials when connecting to SQreamDB, providing a secure and streamlined authentication process.
• Advanced Connectivity –
  • Trino: We are excited to be part of Trino’s connectivity ecosystem, providing users with seamless access to SQreamDB through our JDBC connector. With this integration, Trino users can easily read and write data to SQreamDB using familiar DML commands. Additionally, Trino can query data stored in SQreamDB or integrate external data into SQreamDB.

• Apache Spark: SQreamDB is now integrated with Apache Spark, one of the most widely-used analytics environments in the industry. With our Spark Custom Connector and JDBC, Apache Spark users can seamlessly query SQreamDB to extract data into a DataFrame or write data from a DataFrame into a table within SQreamDB. This integration will greatly streamline data processing and enable Spark users to leverage SQreamDB’s powerful data analytics capabilities.

• Python (PySQream): We’ve upgraded our Python Connector (PySQream) to enable better and simple integration with the Pandas ecosystem, thanks to an updated SQLAlchemy provider. With the ability to read SQreamDB query results into a Pandas DataFrame, data scientists can seamlessly integrate SQream into their MLOps ecosystem, enabling efficient data processing and analysis. Our enhanced Python support further empowers data scientists to leverage SQreamDB’s advanced capabilities in their data-driven workflows.

• Performance improvements – Our team has made significant strides in enhancing the performance of SQreamDB. Our latest improvements include a 70% boost in ORC file ingestion speed, faster performance when querying ZLIB-compressed data, and minimal latency when running DELETE statements (both physical and logical). These optimizations are aimed at providing our customers with faster and more efficient data processing capabilities, enabling them to derive insights from their data in a timely and effective manner.

 

Learn about our latest partnerships with WEKA and VAST Data, and how they facilitate petabyte-scale data processing.

 

Contact us to get the update