Link new docs from README.md (#358)

Disclaimer: to be merged after release 0.22, so the links are valid! Clean up the README, so it does not have redundant information compared to our recently created docs. Maintain content that has not been migrated yet. Logged a follow up ticket to address the debt: #357
astronomer · Jan 10, 2025 · 11ca3e7 · 11ca3e7
1 parent e39607a
commit 11ca3e7
Showing 1 changed file with 11 additions and 101 deletions.
diff --git a/README.md b/README.md
@@ -16,80 +16,27 @@ The minimum requirements for **dag-factory** are:
 - Python 3.8.0+
 - [Apache Airflow®](https://airflow.apache.org) 2.0+
 
-For a gentle introduction, please take a look at our [Quickstart Guide](#quickstart). For more examples, please see the
+For a gentle introduction, please take a look at our [Quickstart Guide](https://astronomer.github.io/dag-factory/latest/getting-started/quick-start-airflow-standalone/). For more examples, please see the
 [examples](/examples) folder.
 
-- [Quickstart](#quickstart)
+- [Quickstart](https://astronomer.github.io/dag-factory/latest/getting-started/quick-start-astro-cli/)
+- [Benefits](#benefits)
 - [Features](#features)
+  - [Dynamically Mapped Tasks](https://astronomer.github.io/dag-factory/latest/features/dynamic_tasks/)
   - [Multiple Configuration Files](#multiple-configuration-files)
-  - [Dynamically Mapped Tasks](#dynamically-mapped-tasks)
   - [Datasets](#datasets)
+  - [Callbacks](#callbacks)
   - [Custom Operators](#custom-operators)
-- [Benefits](#benefits)
 - [Notes](#notes)
   - [HttpSensor (since 1.0.0)](#httpsensor-since-100)
-- [Contributing](#contributing)
-
-## Quickstart
-
-The following example demonstrates how to create a simple DAG using *dag-factory*. We will be generating a DAG with three tasks, where `task_2` and `task_3` depend on `task_1`.
-These tasks will be leveraging the `BashOperator` to execute simple bash commands.
-
-![screenshot](/img/quickstart_dag.png)
-
-(1) To install *dag-factory*, run the following pip command in your [Apache Airflow®](https://airflow.apache.org) environment:
-
-```bash
-pip install dag-factory
-```
-
-(2) Create a YAML configuration file called `config_file.yml` and save it within your dags folder:
-
-```yaml
-example_dag1:
-  default_args:
-    owner: 'example_owner'
-    retries: 1
-    start_date: '2024-01-01'
-  schedule_interval: '0 3 * * *'
-  catchup: False
-  description: 'this is an example dag!'
-  tasks:
-    task_1:
-      operator: airflow.operators.bash_operator.BashOperator
-      bash_command: 'echo 1'
-    task_2:
-      operator: airflow.operators.bash_operator.BashOperator
-      bash_command: 'echo 2'
-      dependencies: [task_1]
-    task_3:
-      operator: airflow.operators.bash_operator.BashOperator
-      bash_command: 'echo 3'
-      dependencies: [task_1]
-```
-
-We are setting the execution order of the tasks by specifying the `dependencies` key.
-
-(3) In the same folder, create a python file called `generate_dags.py`. This file is responsible for generating the DAGs from the configuration file and is a one-time setup.
-You won't need to modify this file unless you want to add more configuration files or change the configuration file name.
-
-```python
-from airflow import DAG  ## by default, this is needed for the dagbag to parse this file
-import dagfactory
-from pathlib import Path
+- [Contributing](https://astronomer.github.io/dag-factory/latest/contributing/howto/)
 
-config_file = Path.cwd() / "dags/config_file.yml"
-dag_factory = dagfactory.DagFactory(config_file)
-
-dag_factory.clean_dags(globals())
-dag_factory.generate_dags(globals())
-```
-
-After a few moments, the DAG will be generated and ready to run in Airflow. Unpause the DAG in the [Apache Airflow®](https://airflow.apache.org) UI and watch the tasks execute!
-
-![screenshot](/img/quickstart_gantt.png)
+## Benefits
 
-Please look at the [examples](/examples) folder for more examples.
+- Construct DAGs without knowing Python
+- Construct DAGs without learning Airflow primitives
+- Avoid duplicative code
+- Everyone loves YAML! ;)
 
 ## Features
 
@@ -103,32 +50,6 @@ If you want to split your DAG configuration into multiple files, you can do so b
     load_yaml_dags(globals_dict=globals(), suffix=['dag.yaml'])
 ```
 
-### Dynamically Mapped Tasks
-
-If you want to create a dynamic number of tasks, you can use the `mapped_tasks` key in the configuration file. The `mapped_tasks` key is a list of dictionaries, where each dictionary represents a task.
-
-```yaml
-...
-  tasks:
-    request:
-      operator: airflow.operators.python.PythonOperator
-      python_callable_name: example_task_mapping
-      python_callable_file: /usr/local/airflow/dags/expand_tasks.py # this file should contain the python callable
-    process:
-      operator: airflow.operators.python_operator.PythonOperator
-      python_callable_name: expand_task
-      python_callable_file: /usr/local/airflow/dags/expand_tasks.py
-      partial:
-        op_kwargs:
-          test_id: "test"
-      expand:
-        op_args:
-          request.output
-      dependencies: [request]
-```
-
-![mapped_tasks_example.png](img/mapped_tasks_example.png)
-
 ### Datasets
 
 **dag-factory** supports scheduling DAGs via [Apache Airflow Datasets](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/datasets.html).
@@ -259,17 +180,6 @@ task_2:
   dependencies: [task_1]
 ```
 
-## Benefits
-
-- Construct DAGs without knowing Python
-- Construct DAGs without learning Airflow primitives
-- Avoid duplicative code
-- Everyone loves YAML! ;)
-
-## Contributing
-
-Contributions are welcome! Just submit a Pull Request or Github Issue.
-
 ## License
 
 To learn more about the terms and conditions for use, reproduction and distribution, read the [Apache License 2.0](https://github.com/astronomer/dag-factory/blob/main/LICENSE).