Azure Synapse Analytics:
Azure Synapse Analytics is a versatile service used for both big data analytics and data warehousing. In addition to performing similar ETL tasks like those in Azure Data Factory, it helps ingest, prepare, and manage data for business intelligence by integrating seamlessly with Power BI and offering its own visualization tools. Synapse provides flexible processing options using either SQL pools for querying structured data or Apache Spark for big data analytics.
The steps for using Azure Synapse Analytics are:
-
Linked Services: Synapse uses Linked Services to connect to external systems like Azure Data Lake, Azure SQL, and other data services, similar to ADF.
-
Pipelines: Synapse orchestrates workflows using Pipelines, which define the steps for moving, transforming, and processing data. Pipelines can handle a wide range of activities, including SQL queries, Apache Spark jobs, and notebooks for data transformation and analysis.
Storage and computing in Azure Synapse Analytics: serverless SQL and dedicated SQL pool are the two main storage options for Azure Synapse Analytics. They both can support external tables which is a Polybase technology that allows us to query and analyze data that is not physically stored inside our storages. Also, Azure Synapse Analytics supports Spark for analyzing big data.
