Optimize Exadata with Gluent’s Live Migration to Snowflake
Gluent’s Live Migration capability allows you to migrate as much or as little of your data warehouse workloads from Exadata to Snowflake. This ranges from no code query offload to full migration. The key is Gluent’s ability to run Oracle data warehouse workloads with Snowflake acting as a super-smart backend server, similar to Exadata Storage Servers, by pushing down query logic to Snowflake, including, SELECT, WHERE and JOIN clauses. Gluent also takes care of synchronising data from Oracle to Snowflake without any separate ELT process.
Exadata Consolidation Trend
Today we are seeing an emerging trend where some Exadata customers are finding Exadata too expensive and inflexible for data warehouse workloads. They can get a lot more value from Exadata by consolidating their OLTP workloads, but to do that they need to release capacity.
Data Warehouse Appliances designed for On Premise
Exadata is undoubtedly the best platform to run Oracle Database workloads. It was originally designed to solve performance problems for Oracle customers who struggled to get good performance for their data warehouse workloads on traditional on-premise infrastructure. Often, they were hampered by shared networks and remote (SAN and NAS) storage. As an appliance, Exadata had dedicated storage with a dedicated high bandwidth network, infiniband at 40Gbs originally. In addition to that some software innovations including Hybrid Column Compression (HCC) which improved performance through reduced I/O and higher density of data in memory, and Smart Scan which enabled resultsets (a table of filtered rows and columns) to be fetched direct from storage rather than pulling entire physical data blocks across the wire for subsequent scanning.
As well as Exadata there were other data warehouse appliance vendors such as Netezza, DATAllegro, Greenplum and Teradata. One thing all these data warehouse appliances have in common is that they answer a question that we are no longer asking! They were a great approach to address the limitations of on-premise systems but now we have Snowflake which is designed to efficiently leverage and orchestrate cloud storage and compute infrastructure to avoid the scale and performance limitations of rigid, fixed capacity, high cost, on-premise compute, memory and storage.
Today, all data warehouse workloads are better run on cloud native, Snowflake.
How do I release Exadata capacity using Gluent’s Live Migration?
Gluent Live Migration allows query, load and transform workloads to be split between Exadata and Snowflake, allowing Exadata capacity to be released systematically. Query workloads can be migrated almost instantly with no code changes, whilst load and transform workloads will require manual or automated code conversion, but this can be broken into manageable steps.
There are three workloads to consider Load, Transform and Query.
Migrating Query Workloads
For Query workloads there are two options.
- Agnostic Query Tools
If you are using SQL agnostic, reporting, dashboards and data science tools, you can simply synchronise the semantic layer(s) which support these to Snowflake using Gluent’s no-ETL incremental offload and point these tools at Snowflake.
2. Oracle SQL and PL/SQL dependent Query Tools
For reporting, dashboard and data science tools which have heavy dependencies on Oracle SQL and PL/SQL or for 3rd party applications which you cannot change, you can continue to connect these to Oracle but use Gluent’s offload and transparent gateway (smart scan) to pushdown work to Snowflake, releasing capacity on Exadata.
Migrating Load and Transform workloads
Load and Transform represents the majority of workload in data warehouse systems. Whilst migrating these workloads will release the most capacity, traditionally this is not usually possible unless all the Load and Transform steps are migrated in together. However, with Gluent we can implement a Live (Incremental) migration where Load and Transform steps are shared between Oracle and Snowflake. This means each step can be migrated from Exadata to Snowflake independently, gradually releasing Exadata capacity and moving towards a position of full migration, if desired once all steps have been ported. I discussed this method in my article Oracle Data Warehouse to Snowflake: Incremental Migration with Gluent
Live Migration Summary
Releasing Exadata capacity allows Oracle customers to leverage Exadata for OLTP workloads, or simply avoid expanding capacity to accommodate increased OLAP data volumes and query workloads.
A traditional full migration is a big undertaking, performing a Live incremental migration allows you to systematically move workloads from Exadata to Snowflake without the need for a high risk, slow return on investment, big bang approach.
With Gluent you can continue to use Oracle and Snowflake together in a hybrid mode which enables Hybrid Transactional and Analytics Processing (HTAP) or to assist in migrating OLAP workloads entirely to Snowflake.
