Azure Data Factory
1. A Cloud Based data integration service
2. Data movement between different data sources
3. Cost-efficient
What is Azure Data Factory
Azure Data Factory (ADF) is a cloud-based ETL and data integration service that enables data to be moved and transformed. Using ADF, you can move data from many different sources and systems. If necessary, you can also transform the data and write it to a central point. This can then be monitored and properly maintained. In addition, ADF can publish data to a database such as Azure Synapse Analytics, which, for example, PowerBI can use to visually display the data.
Through ADF, raw data can eventually be categorized into meaningful data lakes/hubs so that decisions can be made based on the data. A Data Lake is the solution to store all available data, structured and unstructured.
What does it do?
The first step of building a Data Factory is to connect all the sources. A Data Factory can have one or more pipelines. A pipeline is a logical group of activities that perform a task together. The advantage of a pipeline is that the activities can be managed as a group instead of individual activities. The activities in a pipeline can be chained together to run sequentially or independently and in parallel.
Example:
A company wants insight into the figures and needs multiple sources to provide insight into this data. Namely the CRM system, the accounting program and the logistics system. ADF ensures that the data is retrieved from these systems and written to one central point. ADF can link these different resources. In addition, the company can start storing the history of the CRM system with ADF since history is not stored in the CRM system itself.
Azure Data Factory in action
Rockfeather and Azure Data Factory
At Rockfeather we use ADF to connect different sources to one Datahub. This ensures that different source systems are connected in one place in the cloud. In addition, we can ensure that these different source systems can be related to each other through Azure Analysis services. ADF is a data integration solution that does this excellently.
In addition, we work together with Logics Apps in ADF. This combination of ADF together with Logics Apps is a strong combination. For example, ADF can retrieve the data and then use Logic Apps to forward the data to another source. In addition, Logic Apps can help unlock more complex resources.
