Search This Blog

Showing posts with label Azure SQL Data Warehouse. Show all posts
Showing posts with label Azure SQL Data Warehouse. Show all posts

March 05, 2021

Optimizing transactions in Azure Data Warehouse (Synapse)

https://docs.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-best-practices-transactions  

Minimally logged operations

The following operations are capable of being minimally logged:

  • CREATE TABLE AS SELECT (CTAS)
  • INSERT..SELECT
  • CREATE INDEX
  • ALTER INDEX REBUILD
  • DROP INDEX
  • TRUNCATE TABLE
  • DROP TABLE
  • ALTER TABLE SWITCH PARTITION
CTAS and INSERT...SELECT are both bulk load operations

Primary IndexLoad ScenarioLogging Mode
HeapAnyMinimal
Clustered IndexEmpty target tableMinimal
Clustered IndexLoaded rows do not overlap with existing pages in targetMinimal
Clustered IndexLoaded rows overlap with existing pages in targetFull
Clustered Columnstore IndexBatch size >= 102,400 per partition aligned distributionMinimal
Clustered Columnstore IndexBatch size < 102,400 per partition aligned distributionFull

DELETE is a fully logged operation. If you need to delete a large amount of data in a table or a partition, it often makes more sense to SELECT the data you wish to keep, which can be run as a minimally logged operation. To select the data, create a new table with CTAS. Once created, use RENAME to swap out your old table with the newly created table.

UPDATE is a fully logged operation. If you need to update a large number of rows in a table or a partition, it can often be far more efficient to use a minimally logged operation such as CTAS to do so

Both UPDATE and DELETE are fully logged operations and so these undo/redo operations can take significantly longer than equivalent minimally logged operations.


September 09, 2019

Azure Data Warehouse - Materialized View

Materialized View

Materialized Views are similar to Indexed Views in SQL Server. These are used to increase query performance and hide complex query logic from the users.

To enable disabled materialized view, you need to use the "Alter" statement.

ALTER MATERIALIZED VIEW My_Indexed_View REBUILD;

Materialized views are visible within SQL Server Management Studio under the views folder of the Azure SQL Data Warehouse instance

Limitations

  • Materialized views are disabled during an update or delete operations on the base tables.
  • Only Clustered Cloumnstore Index is supported.
  • Only Hash and Round Robin distributions are supported.

Sample Code

CREATE MATERIALIZED VIEW mv_test2
WITH (distribution = hash(i_category_id), FOR_APPEND)
AS
SELECT MAX(i.i_rec_start_date) as max_i_rec_start_date, MIN(i.i_rec_end_date) as min_i_rec_end_date, i.i_item_sk, i.i_item_id, i.i_category_id
FROM syntheticworkload.item i
GROUP BY i.i_item_sk, i.i_item_id, i.i_category_id


For more information, check the following links

https://docs.microsoft.com/en-us/sql/t-sql/statements/create-materialized-view-as-select-transact-sql?view=azure-sqldw-latest


September 05, 2019

Azure SQL Server VS Azure Data Warehouse

Azure SQL Server
  • Database size limit is 4TB
  • Optimized for OLTP loads
  • Cross-database queries are supported
  • Automatic performance tuning of a database is supported


Azure Data Warehouse
  • Supports up to 1 Peta Byte (1024 TB) size.
  • Optimized for OLAP loads
  • MPP (Massively Parallel Processing) system (data is processed on multiple parallel nodes)
  • Polybase support to load data from multiple systems.
  • Cross-database queries are not supported.
  • Manual performance tuning.
  • You can pause and resume the database to reduce cost.