XQ Message

View Original

External Security and Governance for Snowflake

Brian Wane

Stop ransomware, improper sharing, and access to data


Challenges

Snowflake can expose critical data internally and externally with no audit trail of sharing, viewing, or exfiltration

  1. Breaches Happen By Trusting Third Parties with Data Access

    • Organizations lose control of their private data to Snowflake and the applications and identity services they use.

  2. Organizations Risk Losses From Network Security & Governance

    • When data is moved or shared in and out of Snowflake, it loses its protective security measures and can’t be controlled.

  3. No Control of Stolen or Shared Data

    • Organizations have no record of who accessed their data and who is was shared with.

  4. Governance & Compliance Complexity Slows Business

    • Stringent privacy laws intensify the challenges of managing data across global platforms, especially in cloud environments where data localization laws are enforced.


Automated monitoring & external control of every data journey with zero trust data

In today's data-driven world, ensuring the security of sensitive information is paramount. XQ offers a robust solution that significantly enhances the security of your Snowflake database.

Here's how it works and why it makes your data safer:

Unique Encryption for Each Data Item

One of the core features of XQ is the ability to encrypt individual items within a database table uniquely. This means each piece of data is encrypted separately, and specific access controls are set around the encryption keys for each item. Encryption and decryption happen at the edge – on the device you are using to store the data. This device generates the encryption key, encrypts the data, and then stores the key in a keystore along with a set of designated recipients. Crucially, the data never passes through our backend, ensuring that your sensitive information remains in your control.

Self-Hosted Keystore for Complete Control

Each data record can have its own key to prevent lateral movement. The Keystore can be self-hosted or use XQ SaaS, allowing you to manage the encryption keys internally. This provides an additional layer of security since the keys are not stored on an external server, reducing the risk of unauthorized access.

Separation of Data Store and Keystore

What sets XQ apart is the separation of the data store from the keystore. Even if an individual's Snowflake credentials are compromised, the attacker would only access encrypted data. To decrypt each value within the tables, they would need to undergo significant effort and processing, making unauthorized access exceedingly difficult.

Comprehensive Access Logging

Every access attempt to the encrypted data is meticulously logged within the management portal. This generates a detailed record of who accessed the data, what was accessed, where, and when. Such comprehensive logging allows you to monitor and audit real-time data access.

Remote Revocation of Access

You can remotely revoke access to specific keys or users in case of malicious access attempts. This immediate response capability ensures that any potential security breaches are swiftly mitigated.

Summary

By leveraging XQ, you enhance the security of your Snowflake database through unique encryption, separation of data and keys, self-hosted keystore options, detailed access logging, and the ability to revoke access remotely. These features collectively ensure that your data remains secure, even in the event of credential compromise, providing you with peace of mind and robust protection for your sensitive information.


Additional Benefits

  1. Universal Data Protection Across Every Environment

    • XQ guarantees the encryption and control of every data record, eliminating gaps in data security found in current methods.

  2. External Role-Based Access Control (RBAC)

    • By separating tool access from data access, XQ secures and controls your data across all platforms, eliminating any gaps in data governance.

  3. Take Ransomware Extortion Off the Table 

    • XQ technology immediately blocks data access anytime, anywhere—even in the hands of a malicious actor. This guarantees that your data cannot be exploited for extortion, giving you ultimate peace of mind.

  4. Automated Governance & Compliance

    • Automated secrets management and data loss prevention provide a secure chain of custody for easy compliance on every data record.

XQ’s Zero Trust security layer makes data more interoperable and more valuable. By unlocking the possibilities for collaboration, XQ empowers data with automated compliance and self-protection features.

Automated Secret Control

XQ data rights management (DRM) crosses boundaries between environments and automatically creates a unique secure chain of custody for each data record, giving you the proof you need for compliance globally.

Zero Trust Data Sharing 

XQ is a patented Zero Trust Data Platform that separates network and application access from data access, letting you control and monitor who views your data remotely.

Performant 

XQ keeps all your data in Snowflake encrypted until it is being processed when it is decrypted on the fly within an acceptable performance standard.

Cloud Portability 

XQ’s automated external key control allows organizations to safely move data between cloud environments while retaining their security and compliance.

A Step-by-Step Guide

Introduction to XQ Message

XQ Message is an encryption-as-a-service (EaaS) platform designed to provide robust data encryption and decryption capabilities directly on edge devices. It offers a highly secure cybersecurity solution enabling zero-trust data protection (https://en.wikipedia.org/wiki/Zero_trust_security_model) even on devices and applications that cannot natively encrypt or decrypt data. XQ Message ensures data protection by monitoring and controlling all interactions, identifying who accesses the data, their location, and the timing of these interactions.

Full Package Documentation

Prerequisites

Before you begin, ensure you have the following:

  • Python: Supported versions include 3.8, 3.9, 3.10, and 3.11.

  • Conda: This powerful command-line tool for package and environment management runs on Windows, macOS, and Linux.

Installing & Setting Up Conda

Setting Up Snowflake

To use SnowPark, set up a Snowflake account and create a Warehouse, Database, and Schema.

  1. Sign Up for Snowflake: Create an account here. Snowflake offers a 30-day free trial with $400 worth of usage.

  2. Create Warehouse, Database, and Schema: Use the SQL Worksheet in Snowflake to execute the following commands:

    sql

    CREATE WAREHOUSE DWH_EXAMPLE; CREATE DATABASE DB_Example; CREATE SCHEMA DB_Example.Example;

  3. Gather Connection Information: You'll need your ACCOUNT_ID, USERNAME, PASSWORD, and ROLE. The ACCOUNT_ID is found by clicking on your account name in the bottom left corner of the Snowflake interface.


Initializing the XQ SDK

  1. Generate API Keys:

    • Go to your XQ management portal.

    • Create a General key for the XQ framework API.

    • Create a Dashboard key for the XQ dashboard API.

  2. Set Up Environment Variables: Create a .env file in your project directory with the following content:

    #SnowFlake Connection Parameters ACCOUNT_ID='' USERNAME='' PASSWORD='' ROLE='' DATABASE='DB_Example' WAREHOUSE='DWH_EXAMPLE' SCHEMA='Example' # XQ API Key Parameters XQ_API_KEY='' XQ_DASHBOARD_API_KEY=''


Installation

Clone the example repository and set up the Conda environment:

bash

git clone git@github.com:XQ-Message-Inc/pysdk-snowflake-examples.git cd pysdk-snowflake-examples conda env create -f environment.yml conda activate xq-snowflake-examples

Examples

Step 1: Generate an Example Table with Sample Data

Run the following script to create a table with sample data:

bash

python create_example.py

This script creates a table with columns ID, FirstName, LastName, Address, and PhoneNumber, and populates it with 10 random entries by default.


Step 2: Encrypt Data within the Example Table

You can encrypt individual rows using either One-Time Pad (OTP) or AES256. Run the respective scripts:

  • One-Time Pad:

    bash

    python encryption_otp.py

  • AES:

    bash

    python encryption_aes.py

These scripts handle authentication with XQ, generate encryption keys, and encrypt each row in the table. The encrypted data is then written back to the Snowflake table.

Step 3: Decrypt Data within the Example Table

To decrypt the data, use the appropriate script based on the encryption method used:

  • One-Time Pad:

    bash

    python decryption_otp.py

  • AES:

    bash

    python decryption_aes.py

These scripts authenticate with XQ, retrieve the encrypted data, and decrypt each row, printing the decrypted values to the console.

Conclusion

By following these steps, you can securely encrypt and decrypt data within Snowflake using XQ Message's Python SDK. This setup ensures that your data remains protected with robust encryption methods, leveraging the power of Snowflake and XQ's secure platform. For further details, refer to the full package documentation available in the XQ Message repository.