Optimizing Business Efficiency with Snowflake

Decision-makers in finance, healthcare, retail, education and other verticals rely on the accurate processing and analyzing of data to direct corporate strategy. The need to analyze Big Data more efficiently becomes increasingly significant as the volume of data grows. Here’s when Snowflake helped bring that efficiency.

Snowflake’s computing and storage tiers are entirely independent of one another, and the platform’s two tiers are almost instantly flexible. With Snowflake, there is no longer any need for in-depth resource planning, agonizing over workload schedules, or forbidding the addition of new workloads out of concern for disc and CPU capacity restrictions.

The Power of Cloud

Source Credit: Snowflake

Snowflake can almost instantaneously scale as a cloud data platform to accommodate anticipated, unforeseen, or planned growth. This means that as your needs change over time, you pay for a growing and shrinking quantity of storage and computation rather than a fixed, limited amount.

The cloud creates access to near-infinite, low-cost storage, the capability to scale up/down, outsourcing the challenging operations tasks of data warehousing management and security to the cloud vendor and plus potential to pay for only the storage and computing resources actually used, when you use them.

Having worked numerous times over the years with everything from Hadoop to Teradata/Oracle as well as having been deeply involved on migration projects moving workloads from on-premise environments to the cloud and the existing data ingestion process was stable and mature. Daily, an ETL script loads the raw CSV/TXT/JSON file from the file system and inserts the file’s contents into a SQL table which is stored in ORC/CSV format with snappy compression.

In order to avoid re-implementation of the ETL process, the first constraint was the cloud data warehouse needed to support the ORC file format. There are two cloud data warehouses that support ORC file format i.e. Snowflake and Amazon Redshift Spectrum. As external files located in Amazon S3, both Snowflake and Redshift Spectrum permit queries on ORC files. However, Snowflake edged out Redshift Spectrum for its expertise to also load and transform ORC data files directly into Snowflake.

Read more on our blogs Cluster Analysis 101

Features of Snowflake Architecture

• Snowflake stores semi-structured data such as JSON, Avro, ORC, Parquet, and XML alongside your relational data. Query all data with standard i.e. ACID-compliant SQL, and dot notation.
• Support concurrent use cases with independent virtual warehouses (compute clusters) that reference your common data
• Maintain your investment and assets in the skills and tools a user already rely on for your data analytics.
• Easily forge one-to-one, one-to-many, and many-to-many data sharing relationships, so your business units, subsidiaries, and partners can securely query read-only, centralized data.

Optimizing Business Efficiency with Snowflake

Snowflake accredit the data-driven enterprise with instant elasticity, secure data sharing and per-second pricing. Its built-for-the-cloud architecture combines the power of data warehousing, the flexibility of big data platforms, and the elasticity of the cloud. Snowflake is an APN Advanced Technology Partner and has achieved Data & Analytics Competency.

The Snowflake frame is a modern data warehouse which is effective, affordable, and accessible to all data users within the organization. In the next part, we’ll introduce the Architecture, Pros/Cons and Table Design Considerations.