Add declarative configuration to define custom roles

[ketozhang]

Description

Add declarative configuration to define custom roles in either airflow.cfg or webserver_config.py.

Use case/motivation

Currently the only way to define custom roles is through the CLI or UI. This isn't ideal for those deploying configuration through a repository (e.g., GitOps).

It seems like this feature is already possible with webserver_config.py, however not documented here. If so, I will change the request...
https://airflow.apache.org/docs/apache-airflow/stable/security/webserver.html

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

Sure. If you have a good idea how to do it in a robust way - proposing a solution, especially one that will not overwrite existing DB and will allow to both declaratively upload roles from a source and manually modify them, and keep those local modificiattions - PRs are most welcome.

Just make sure when you submit the PR to propose a solution for this conundrum where you have both manually modifiable data and uploadable from the externally depoyable sources in declarative way. So far all the efforts failed on that step, but maybe you will be able to propose something.

BTW. once you find a solution, also similar thing should be done from connections, users and variables - those suffer from the same issue and if you find and get approval to solve it for roles, applying similar approach for the others is a natural next step.

[ketozhang]

Thanks and yep, it's a classic problem most devops folks have to deal with. It's heavily dependent if you want to lock down configs post-deploy and if DB should be overwritten during a redeploy.

What's currently being done for connections and variables to solve that issue? They both already have declarative equivalents.

So far all the efforts failed on that step

How have they failed?

[potiuk]

How have they failed?

To solve all the edge cases and the fact that users can modify the DB after it has been created and how to consistently mantain source of the truth.t But maybe your proposal will fix it. Looking forward to it.

[potiuk]

What's currently being done for connections and variables to solve that issue? They both already have declarative equivalents.

No - they don't - same problem. When you allow to edit data over the UI and want to also upload it at the deployment time, you need to solve the problem of data being modified both manually and through declarative sources at deplpyment time

[shahar1]

@ketozhang Would you still like to work on this issue?

[ketozhang]

I don't have a implementation proposal yet. Someone else can take it if they've got ideas.

---

Well I do have one idea... env vars. Airflow does allow many configs to be set/overridden with env vars (I guess it's a simple if env var exist, don't use DB values).

The value of the role env var would be JSON string. Here's an example without proper formatting...

{
  "roles": {
    "PythonTester": [
      {
        "permission": "ACTION_CAN_EDIT",
        "resource": "RESOURCE_DAG"
      }
    ]
  }
}

A few caveats:

• You cannot modify the roles of the same name in the UI and env-vars. Precedence goes to env-vars. Different name is OK.
  • UI should show env-vars defined roles to be grayed-out (i.e., read-only)
• You cannot use env-vars to define custom roles with DAG-level permissions. Instead, this is declaratively set by the DAG file (DAG(access_control={...}))