DP-750: Lakeflow Jobs Explained and with Real Exam Questions

When preparing for DP-750: Microsoft Certified: Azure Databricks Data Engineer Associate, you need to understand not only how to process data, but also how to orchestrate data workloads.

In Azure Databricks, this orchestration layer is mainly handled by Lakeflow Jobs.

Lakeflow Jobs are used to run and coordinate data processing workflows. A job can contain one or more tasks, such as notebook tasks, pipeline tasks, Python script tasks, and other task types. Tasks can have dependencies on other tasks, which allows you to build workflows with execution order, conditional logic, and dependencies.

This topic appears many times in the DP-750 question bank, especially in questions about file-arrival triggers, multi-step pipeline tasks, repair runs, concurrency control, retry policies, and notifications.

---

1. What is Lakeflow Jobs?

Lakeflow Jobs is the Azure Databricks workflow orchestration service.

You can use Lakeflow Jobs to run:

• notebooks
• Python scripts
• SQL tasks
• Lakeflow Spark Declarative Pipelines
• dbt tasks
• JAR tasks
• multi-task workflows

A Lakeflow Job usually contains:

• one or more tasks
• compute for each task
• dependencies between tasks
• parameters
• schedules or triggers
• retry policies
• notifications
• monitoring and run history

A job can be created and run from the Databricks UI, Databricks CLI, REST API, or Declarative Automation Bundles. A job task contains the logic to run, while the compute resource runs that logic. The compute can be serverless compute, classic jobs compute, or all-purpose compute.

For DP-750, the most important idea is:

> Lakeflow Jobs = orchestration > Lakeflow Spark Declarative Pipelines = declarative data pipeline processing

They are related, but they are not the same thing.

---

2. Why use Lakeflow Jobs?

In real data engineering projects, pipelines usually contain multiple steps.

For example:

1. Ingest raw files.
2. Clean and validate data.
3. Transform data.
4. Publish curated tables.
5. Run quality checks.
6. Notify operators if something fails.

If you put all logic into one large notebook, the workflow becomes difficult to maintain, retry, monitor, and recover.

Lakeflow Jobs help you separate a workflow into smaller tasks. This makes the pipeline easier to operate.

For DP-750, when the question says:

• orchestrate multi-step workflows
• define execution order
• configure dependencies
• retry failed steps
• notify operators
• avoid rerunning successful tasks
• prevent downstream tasks from running after upstream failure

you should think of Lakeflow Jobs.

---

3. Tasks and dependencies

A task is one unit of work inside a Lakeflow Job.

For example:

• Task1: ingest raw data
• Task2: cleanse data
• Task3: create curated tables

Tasks can depend on upstream tasks. Azure Databricks represents dependencies visually in the job DAG. Upstream tasks run before downstream tasks, and Databricks runs as many tasks in parallel as possible when dependencies allow it.

This is important for the exam.

If the question says:

> If Task1 fails, Task2 must not run.

The normal design is:

• Task2 depends on Task1.
• Task2 runs only if Task1 succeeds.

In Lakeflow Jobs, the default dependency behavior is All succeeded. This means all dependencies must run and succeed before the downstream task runs. If the condition is not met, the downstream task is marked as upstream failed.

---

4. Separate tasks vs one large notebook

A common DP-750 exam pattern is about whether to use one notebook for ingestion, cleansing, and curation, or separate tasks.

For production pipelines, separate tasks are usually better.

For example:

|Pipeline step|Recommended task design| |---|---| |Ingestion|Task 1| |Cleansing|Task 2| |Curation|Task 3|

This design gives you:

• clearer execution order
• better monitoring
• easier troubleshooting
• task-level retry
• better recovery with repair runs
• less duplicated logic

In DP-750, when the question says:

> Each task must contain only the appropriate logic for its step in the pipeline.

the answer should be:

> Create separate tasks for ingestion, cleansing, and curation.

---

5. Job triggers

A trigger controls when a job starts.

Azure Databricks supports several trigger types for Lakeflow Jobs, including scheduled triggers, table update triggers, file arrival triggers, model update triggers, continuous triggers, manual runs, and external orchestration.

The most important trigger types for DP-750 are:

|Requirement|Trigger| |---|---| |Run at a fixed time|Scheduled trigger| |Run when new files arrive|File arrival trigger| |Run when source tables update|Table update trigger| |Keep job always running|Continuous trigger| |Run only when manually started|Manual trigger|

File arrival trigger

A file arrival trigger starts a job when new files arrive in a monitored Unity Catalog storage location.

Use it when:

• files arrive unpredictably
• new files should be processed quickly
• you do not want compute running when no files are available

In DP-750, if the question says:

> New records must be processed as soon as they become available.

or:

> New files arrive at unpredictable intervals.

the answer is often:

> File arrival trigger

Continuous trigger

A continuous trigger keeps the job always running by triggering another run whenever a job run completes or fails.

Use it when the job should keep running continuously.

Do not confuse this with file arrival. If the requirement says the job should not consume compute when no data is available, file arrival is usually better.

---

6. Retry policy

A retry policy automatically reruns a task after it fails.

Lakeflow Jobs allow retries for tasks. Retries specify how many times a task should be rerun if it fails. This is useful because many failures are transient and can be resolved by restarting the task.

For DP-750:

> transient failure + automatic retry = configure a retry policy

Retry policy is usually configured at the task level when the question is about a specific task.

For example:

> If Task1 fails, Task1 should retry automatically, and Task2 should not run.

The answer pattern is:

• configure a retry policy for Task1
• block downstream tasks if Task1 fails

---

7. Repair run

A repair run is used after a multi-task job has failed.

It reruns only the unsuccessful tasks and any dependent tasks. Successful tasks are not rerun, which reduces recovery time and compute cost.

This is a very important DP-750 concept.

Use a repair run when:

• a multi-task job failed
• some tasks already completed successfully
• you want to avoid rerunning successful tasks
• you want to retry only the failed part of the workflow

For example:

|Situation|Best action| |---|---| |Final task failed after a long job run|Repair the current job run| |Task1 failed and downstream tasks did not run|Fix issue, then repair run| |Single-task job failed|Run the job again| |Need automatic retry during execution|Task retry policy|

A repair run is not exactly the same as a retry policy.

|Feature|When it is used| |---|---| |Retry policy|During job execution, after task failure| |Repair run|After a failed or canceled multi-task job run|

Databricks also notes that a repair reruns unsuccessful tasks from the beginning, so task logic should be idempotent if duplicate writes are a concern.

---

8. Concurrency control

Sometimes a scheduled job takes longer than expected.

For example:

• Job1 runs every hour.
• One run takes more than one hour.
• A new run starts before the previous run finishes.
• Overlapping runs may corrupt data.

In this case, you need to prevent concurrent runs.

The Lakeflow Jobs trigger documentation says that by default only one run of a job can be active at a time, but the maximum concurrency can be changed in advanced settings. Runs are skipped when they exceed the configured maximum concurrency for a job.

For DP-750, the expected answer pattern is usually:

• limit concurrent runs to one
• queue the new run

This prevents overlapping execution and avoids data corruption.

---

9. Notifications

Lakeflow Jobs can send notifications for job and task events.

For job-level notifications, Azure Databricks supports event types such as start, success, failure, duration warning, and streaming backlog.

There is an important exam detail:

> Job-level notifications are not sent when failed tasks are retried. > To receive a failure notification after every failed task, use task notifications.

This is why some DP-750 questions expect task failure as the notification trigger, especially when only failures in critical tasks should send notifications.

Use this logic:

|Requirement|Notification pattern| |---|---| |Notify when the whole job fails|Job failure notification| |Notify only when a critical task fails|Task failure notification| |Avoid notifications for non-critical task failures|Configure notifications only on critical tasks| |Notify after every failed task|Task notifications|

---

10. DP-750 decision table for Lakeflow Jobs

|Scenario in the question|Best answer pattern| |---|---| |Multi-step workflow|Lakeflow Jobs| |Ingestion, cleansing, curation steps|Separate tasks| |Define execution order|Task dependencies| |Prevent Task2 from running if Task1 fails|Downstream dependency / block downstream tasks| |Automatically retry a failed task|Task-level retry policy| |Avoid rerunning successful tasks after failure|Repair run| |New files arrive unpredictably|File arrival trigger| |Process data as soon as files arrive|File arrival trigger| |Run job at fixed time|Scheduled trigger| |Keep job always running|Continuous trigger| |Prevent overlapping runs|Limit concurrent runs to one| |Notify when job fails|Job failure notification| |Notify only when critical task fails|Task failure notification|

---

Real Exam Questions

Question 38

You have an Azure Databricks workspace that uses serverless compute.

You need to ingest data by using Lakeflow Jobs. New records must be processed as soon as they become available.

Which type of job trigger should you use for the ingestion?

A. manual B. file arrival ✅ Correct Answer C. scheduled D. continuous

---

Question 60

Useful case information

Contoso plans to implement scalable data pipeline orchestration.

Contoso identifies the following pipeline deployment and operation requirements:

• Orchestrate multi-step ingestion and transformation workflows.
• Define a clear execution order and dependencies.
• Automatically retry failed steps and notify operators.
• Schedule ingestion and transformation workloads consistently.

Contoso also needs to build a new telemetry pipeline that ingests raw events from the event hubs, cleanses the data, and publishes curated tables to Unity Catalog.

You need to develop the task logic for a new job in Lakeflow Jobs that processes telemetry data.

Each task must contain only the appropriate logic for its step in the pipeline. The solution must support the planned changes and meet the data ingestion and processing requirements.

What should you do?

A. Use a single Databricks notebook task that performs ingestion, cleansing, and curation in one script.

B. Create separate tasks for ingestion, cleansing, and curation. ✅ Correct Answer

C. Create three tasks that each contains the identical logic and use task retries.

D. Use a single SQL task that performs ingestion, cleansing, and curation by running MERGE commands.

---

Question 64

You have an Azure Databricks workspace that is enabled for Unity Catalog.

You have a complex job named Job1 that contains eight tasks. Job1 takes multiple hours to complete.

During the last job run, the final task fails due to a transient issue.

You need to retry the last task without rerunning tasks that have already completed.

What should you do?

A. Repair the current job run. ✅ Correct Answer

B. Update the job parameters.

C. Disable and reenable the job schedule.

D. Restart Job1.

---

Question 65

You have an Azure Databricks workspace that contains a job in Lakeflow Jobs named Job1.

Job1 runs every hour.

Occasionally, the job run takes longer than one hour to complete. Overlapping runs must be prevented to avoid data corruption.

You need to configure the job scheduling behavior.

What should you configure?

|Area|Answer| |---|---| |Concurrency setting|Limit concurrent runs to one ✅ Correct Answer| |Execution behavior|Queue the new run ✅ Correct Answer|

---

Question 69

You have an Azure Databricks workspace that contains a job in Lakeflow Jobs named Job1.

Job1 processes raw data files stored in Azure Storage.

New files arrive at unpredictable intervals.

You need to ensure that Job1 starts automatically when new files arrive and does NOT consume compute resources when no data is available.

Which type of job trigger should you use?

A. manual

B. file arrival ✅ Correct Answer

C. continuous

D. scheduled

---

Question 72

You have an Azure Databricks job named Job1 that contains an ingestion task named Task1 and transformation task named Task2.

You need to ensure that if Task1 fails, the task retries automatically, and Task2 is prevented from running.

How should you configure Job1?

|Area|Answer| |---|---| |Failure handling behavior|Block downstream tasks ✅ Correct Answer| |Retry configuration|Configure a retry policy for Task1 ✅ Correct Answer|

---

Question 74

You have an Azure Databricks workspace that contains a job in Lakeflow Jobs named Job1.

Job1 contains three tasks named Task1, Task2, and Task3.

If Task1 fails, Task2 and Task3 must be prevented from running. Successfully completed tasks must NOT rerun during recovery.

You need to configure Job1 to support controlled failure handling and recovery.

What should you configure?

|Area|Answer| |---|---| |Failure handling behavior|Fail the job if Task1 fails ✅ Correct Answer| |Recovery mechanism|Configure a repair run ✅ Correct Answer|

---

Question 75

You have an Azure Databricks workspace that contains a job in Lakeflow Jobs named Job1. Job1 contains multiple tasks.

Failures of non-critical tasks must be logged but must NOT trigger notifications. Notifications must be triggered only when critical tasks have failed, and Job1 has completed.

You need to configure the job alerting behavior.

What should trigger a notification?

A. task success

B. a job failure

C. a task failure ✅ Correct Answer

D. job success

---

Question 79

You have an Azure Databricks workspace that contains a job in Lakeflow Jobs named Job1.

Job1 runs on a schedule and processes business-critical data.

You need to ensure that you receive a notification if Job1 fails.

What should you do?

A. Add a SQL alert task to Job1.

B. Create an Azure Monitor alert rule based on the metrics of Job1.

C. Send the job run results of Job1 to Log Analytics.

D. Configure a failure notification to be sent for the job runs of Job1. ✅ Correct Answer

---

Question 82

You have an Azure Databricks workspace that contains a job in Lakeflow Jobs named Job1. Job1 contains multiple tasks.

Some tasks are business-critical, and others are not.

You need to configure alerting so that:

• failures in non-critical tasks do not send notifications

• notifications are sent only for failures in the critical parts of the workflow

What should trigger a notification?

A. task success

B. job success

C. a task failure ✅ Correct Answer

D. a job failure

---

Key takeaways

For DP-750, Lakeflow Jobs questions are usually about workflow orchestration.

Remember these patterns:

• Multi-step pipeline → Lakeflow Jobs

• Ingestion, cleansing, curation → separate tasks

• Execution order → task dependencies

• Task1 fails, Task2 should not run → block downstream tasks

• Transient task failure → retry policy

• Final task failed, earlier tasks succeeded → repair run

• New files arrive unpredictably → file arrival trigger

• Scheduled job overlaps → limit concurrent runs to one

• Only critical task failures should notify → task failure notification

• Whole job failure should notify → job failure notification

If you understand these decision patterns, most DP-750 Lakeflow Jobs questions become straightforward.