50+ most common job interview questions
Tell me about yourself.
Walk me through your resume.
How did you hear about this position?
Why do you want to work at this company?
Why do you want this job?
Why should we hire you?
What can you bring to the company?
What are your greatest strengths?
What do you consider to be your weaknesses?
What is your greatest professional achievement?
Tell me about a challenge or conflict you’ve faced at work, and how you dealt with it.
Tell me about a time you demonstrated leadership skills.
What’s a time you disagreed with a decision that was made at work?
Tell me about a time you made a mistake.
Tell me about a time you failed.
Why are you leaving your current job?
Why were you fired?
Why was there a gap in your employment?
Can you explain why you changed career paths?
What’s your current salary?
What do you like least about your job?
What are you looking for in a new position?
What type of work environment do you prefer?
What’s your work style?
What’s your management style?
How would your boss and coworkers describe you?
How do you deal with pressure or stressful situations?
What do you like to do outside of work?
Are you planning on having children?
How do you stay organized?
How do you prioritize your work?
What are you passionate about?
What motivates you?
What are your pet peeves?
How do you like to be managed?
Do you consider yourself successful?
Where do you see yourself in five years?
How do you plan to achieve your career goals?
What are your career aspirations?
What’s your dream job?
What other companies are you interviewing with?
What makes you unique?
What should I know that’s not on your resume?
What would your first few months look like in this role?
What are your salary expectations?
What do you think we could do better or differently?
When can you start?
Are you willing to relocate?
How many tennis balls can you fit into a limousine?
If you were an animal, which one would you want to be?
Sell me this pen.
Is there anything else you’d like us to know?
Do you have any questions for us?
Top 5 Apache AIRFLOW Interview Questions | Advanced Apache AIRFLOW
airflow interview questions
1. How will you describe Airflow?
Apache Airflow is referred to an open-source platform that is used for workflow management. This one is a data transformation pipeline Extract, Transform, Load (ETL) workflow orchestration tool. It initiated its operations back in October 2014 at Airbnb. At that time, it offered a solution to manage the increasingly complicated workflows of a company. This Airflow tool allowed them to programmatically write, schedule and regulate the workflows through an inbuilt Airflow user interface.
2. What are the problems resolved by Airflow?
Some of the issues and problems resolved by Airflow include:
- Maintaining an audit trail of every completed task
- Scalable in nature
- Creating and maintaining a relationship between tasks with ease
- Comes with a UI that can track and monitor the execution of the workflow and more.
3. What are some of the features of Apache Airflow?
Some of the features of Apache Airflow include:
- It helps schedule all the jobs and their historical status
- Helps in supporting executions through web UI and CRUD operations on DAG
- Helps view Directed Acyclic Graphs and the relation dependencies
4. How does Apache Airflow act as a Solution?
Airflow solves a variety of problems, such as:
- Failures: This tool assists in retrying in case there is a failure.
- Monitoring: It helps in checking if the status has been succeeded or failed.
- Dependency: There are two different types of dependencies, such as:
- Data Dependencies that assist in upstreaming the data
- Execution Dependencies that assist in deploying all the new changes
- Scalability: It helps centralise the scheduler
- Deployment: It is useful in deploying changes with ease
- Processing Historical Data: It is effective in backfilling historical data
5. Define the basic concepts in Airflow.
Airflow has four basic concepts, such as:
- DAG: It acts as the order’s description that is used for work
- Task Instance: It is a task that is assigned to a DAG
- Operator: This one is a Template that carries out the work
- Task: It is a parameterized instance
6. Define integrations of the Airflow.
Some of the integrations that you’ll find in Airflow include:
- Apache Plg
- Amazon EMR
- Kubernetes
- Amazon S3
- AWS Glue
- Hadoop
- Azure Data Lake
7. What do you know about the command line?
The command line is used to run Apache Airflow. There are some significant commands that everybody should know, such as:
- Airflow run is used for running a task
- Airflow show DAG is used for showcasing tasks and their dependencies
- Airflow task is used for debugging tasks
- Airflow Webserver is used for beginning the GUI
- Airflow backfill is used for running a specific part of DAG
8. How would you create a new DAG?
There are two different methods to create a new DAG, such as:
- By writing a Python code
- By testing the code
9. What do you mean by Xcoms?
Cross Communication (XComs) is a mechanism that allows tasks to talk to one another. The default tasks get isolated and can run on varying machines. They can be comprehended by a Key and by dag_id and task_id.
10. Define Jinja Templates.
Jinja templates assist by offering pipeline authors that contain a specific set of inbuilt Macros and Parameters. Normally, it’s a template that contains Expressions and Variables.
Airflow Interview Questions for Experienced
1. Explain the design of workflow in Airflow.
To design workflow in this tool, a Directed Acyclic Graph (DAG) is used. When creating a workflow, you must contemplate how it could be divided into varying tasks that can be independent. And then, the tasks are combined into a graph to create a logical whole.
The overall, comprehensive logic of the workflow is dependent on the graph’s shape. An Airflow DAG can come with multiple branches, and you can select the ones to follow and the ones to skip during the execution of the workflow.
Also:
- Airflow can be halted, completed and can run workflows by resuming from the last unfinished task.
- It is crucial to keep in mind that Airflow operators can run multiple times when designing. Every task should be independent and capable of being performed several times without leading to any unintentional consequences.
2. What do you know about Airflow Architecture and its components?
There are four primary Airflow components, such as:
- Web Server
This one is an Airflow UI that is developed on the Flask and offers an overview of the complete health of a variety of DAGs. Furthermore, it also helps visualise an array of components and states of each DAG. For the Airflow setup, the web server also lets you manage different configurations, roles, and users. - Scheduler
Every ‘n’ second, the scheduler navigates through the DAGs and helps schedule the tasks that have to be executed. The scheduler has an internal component as well, which is known as the Executor. Just as the name suggests, it helps execute the tasks, and the scheduler orchestrates all of them. In Airflow, you’ll find a variety of Executors, such as KubernetesExecutor, CeleryExecutor, LocalExecutor and SequentialExecutor.
- Worker
Basically, workers are liable for running those tasks that the Executor has provided to them. - Metadata Database
Airflow supports an extensive range of metadata storage databases. These comprise information regarding DAGs and their runs along with other Airflow configurations, such as connections, roles and users. Also, the database is used by the webserver to showcase the states and runs of the DAGs.
3. Define the types of Executors in Airflow.
The Executors, as mentioned above, are such components that execute tasks. Thus, Airflow has a variety of them, such as:
- SequentialExecutor
SequentialExecutor only executes one task at a time. Herein, the workers and the scheduler both use a similar machine. - KubernetesExecutor
This one runs every task in its own Kubernetes pod. On-demand, it spins up the worker pods, thus, enabling the efficient use of resources - LocalExecutor
In most ways, this one is the same as the SequentialExecutor. However, the only difference is that it can run several tasks at a time. - CeleryExecutor
Celery is typically a Python framework that is used for running distributed asynchronous tasks. Thus, it has been a page of Airflow for a long time now. CeleryExecutors come with a fixed number of workers that are always on the standby to take tasks whenever available.
4. Can you define the pros and cons of all Executors in Airflow?
Here are the pros and cons of Executors in Airflow.
| Executors | Pros | Cons |
| SequentialExecutor |
|
|
| LocalExecutor |
|
|
| CeleryExecutor |
|
|
| KubernetesExecutor |
|
|
5. How can you define a workflow in Airflow?
To define workflows in Airflow, Python files are used. The DAG Python class lets you create a Directed Acyclic Graph, which represents the workflow.
from Airflow.models import DAG
from airflow.utils.dates import days_ago
args = {
'start_date': days_ago(0),
}
dag = DAG(
dag_id='bash_operator_example',
default_args=args,
schedule_interval='* * * * *',
)You can use the beginning date to launch any task on a certain date. The schedule interval also specifies how often every workflow is scheduled to run. Also, ‘* * * * *’ represents that the tasks should run each minute.
6. Can you tell us some Airflow dependencies?
Some of the dependencies in Airflow are mentioned below:
freetds-bin \
krb5-user \
ldap-utils \
libffi6 \
libsasl2-2 \
libsasl2-modules \
locales \
lsb-release \
sasl2-bin \
sqlite3 \
7. How can you restart the Airflow webserver?
The Airflow web server can be restarted through data pipelines. Also, the backend process can be started through this command:
airflow webserver -p 8080 -B true
8. How can you run a bash script file?
The bash script file can run with this command:
create_command = """
./scripts/create_file.sh
"""
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)9. How would you add logs to Airflow logs?
We can add logs either through the logging module or by using the below-mentioned command:
import
dag = xx
def print_params_fn(**KKA):
import logging
logging.info(KKA)
return None
print_params = PythonOperator(task_id="print_params",
python_callable=print_params_fn,
provide_context=True,
dag=dag)10. How can you use Airflow XComs in Jinja templates?
We can use Airflow XComs in Jinja templates through this command:
SELECT * FROM {{ task_instance.xcom_pull(task_ids='foo', key='Table_Name') }}Conclusion
Once you’re backed up by the right type of preparation material, cracking an interview becomes a seamless experience. So, without further ado, refer to these Airflow interview questions mentioned above and sharpen your skills substantially.
How does Apache Airflow acts as Solution?Airflow solves problems like:
Airflow consists of 4 concepts:
- DAG – acts as a description of the order used for work.
- Operator – acts as a Template for carrying out work.
- Task – acts as a parameterized instance.
- Task Instance – acts as a task which is assigned to a DAG.
What is Apache Airflow?
Apache Airflow is an open source workflow management that helps us by managing workflow Orchestration with the help of DAGs(Directed Acyclic Graphs).It is written in Python language and the workflow are created through python scripts.Airflow is designed by the principle of Configuration as Code. Apache Airflow began at Airbnb for managing platform and the company’s icreasing complex workflows.It is known as data transformation pipeline Extract,Transform,Load(ETL) workflow Orchestration Tool.
- Web Server – used for tracking the status of our jobs and in reading logs from a remote File Store.
- Scheduler – used for scheduling our jobs and is a multithreaded python process which use DAGb object.
- Executor – used for getting the tasks done.
- Metadata Database – used for storing the Airflow States.
