++
1 Playground¶
+Content
+-
+
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
1.1 Inline markup¶
+text - emphasis (italics),
+ text - emphasis (boldface), and
+ text - code samples.
1.2 List and quotes blocks¶
+-
+
This is a bulleted list.
+ It has two items, the second + item uses two lines.
+
-
+
This is a numbered list.
+ It has two items too.
+ This is a numbered list.
+ It has two items too.
+
-
+
this is
+ a list
+-
+
with a nested list
+ and some subitems
+
+ and here the parent list continues
+
1.3 Definitions list¶
+-
+
- term (up to a line of text)
Definition of the term, which must be indented
+and can even consist of multiple paragraphs
+
+ - next term
Description.
+
+
1.4 Parameter list¶
+-
+
- Date +
2001-08-16
+
+ - Version +
1
+
+ - Authors +
-
+
Me
+ Myself
+ I
+
+ - Indentation +
Since the field marker may be quite long, the second + and subsequent lines of the field body do not have to line up + with the first line, but they must be indented relative to the + field name marker, and they must line up with each other.
+
+ - Parameter i +
integer
+
+
1.5 Option list¶
+-
+
- -a +
Output all.
+
+ - -b +
Output both (this description is + quite long).
+
+ - -c arg +
Output just arg.
+
+ - --long +
Output all day long.
+
+ - -p +
This option has two paragraphs in the description. + This is the first.
+This is the second. Blank lines may be omitted between + options (as above) or left in (as here and below).
+
+ - --very-long-option +
A VMS-style option. Note the adjustment for + the required two spaces.
+
+ - --an-even-longer-option +
The description can also start on the next line.
+
+ - -2, --two +
This option has two variants.
+
+ - -f FILE, --file=FILE +
These two options are synonyms; both have + arguments.
+
+ - /V +
A VMS/DOS-style option.
+
+
1.6 Literal blocks¶
+This is a typical paragraph. An indented literal block follows.
+for a in [5,4,3,2,1]: # this is program code, shown as-is
+ print a
+print "it's..."
+# a literal block continues until the indentation ends
+This text has returned to the indentation of the first paragraph, + is outside of the literal block, and is therefore treated as an + ordinary paragraph.
+1.7 Quoted literal blocks¶
+John Doe wrote:
+>> Great idea!
+>
+> Why didn't I think of that?
+You just did! ;-)
+1.8 Doc test blocks¶
+This is an ordinary paragraph.
+>>> print 'this is a Doctest block'
+this is a Doctest block
+The following is a literal block:
+>>> This is not recognized as a doctest block by
+reStructuredText. It *will* be recognized by the doctest
+module, though!
+1.9 Grid tables¶
+1.10 Footnotes¶
+1 is a reference to footnote 1, and 3 is a reference to + footnote 2.
+ +4 is a reference to footnote 3.
+2 will be “2” (manually numbered), + 6 will be “3” (anonymous auto-numbered), and + 5 will be “1” (labeled auto-numbered).
+-
+
- 2 +
This footnote is labeled manually, so its number is fixed.
+
+ - 5 +
This autonumber-labeled footnote will be labeled “1”. + It is the first auto-numbered footnote and no other footnote + with label “1” exists. The order of the footnotes is used to + determine numbering, not the order of the footnote references.
+
+ - 6 +
This footnote will be labeled “3”. It is the second + auto-numbered footnote, but footnote label “2” is already used.
+
+
Here is a citation reference: [CIT2002].
+-
+
- CIT2002 +
This is the citation. It’s just like a footnote, + except the label is textual.
+
+
The 
symbol must be used on containers used to
+ dispose of medical waste.
Para.
++
Para.
+1.11 Admonitions¶
+Attention
+Beware killer rabbits!
+Caution
+Beware killer rabbits!
+Danger
+Beware killer rabbits!
+Error
+Beware killer rabbits!
+Hint
+Beware killer rabbits!
+Important
+Beware killer rabbits!
+Note
+Beware killer rabbits!
+Tip
+Beware killer rabbits!
+Warning
+Beware killer rabbits!
+Note
+This is a note admonition. + This is the second line of the first paragraph.
+-
+
The note contains all indented body elements + following.
+ It includes this bullet list.
+
New in version 2.5: The spam parameter.
+Changed in version 2.5: The spam parameter.
+Deprecated since version 2.5: The spam parameter.
+See also
+-
+
- Module
zipfile Documentation of the
+zipfilestandard module.
+ - GNU tar manual, Basic Tar Format
Documentation for tar archive files, including GNU tar extensions.
+
+
AAAAAA
+-
+
- environment
A structure where information about all documents under the root is + saved, and used for cross-referencing. The environment is pickled + after the parsing stage, so that successive runs only need to read + and parse new and changed documents.
+
+ - source directory
The directory which, including its subdirectories, contains all + source files for one Sphinx project.
+
+
1.12 Figure¶
+
+ This is the caption of the figure (a simple paragraph).¶
+The legend consists of all elements after the caption. In this + case, the legend consists of this paragraph and the following + table:
+Symbol |
+ Meaning |
+
|---|---|
+ |
+ Campground |
+
+ |
+ Lake |
+
+ |
+ Mountain |
+
1.13 Topic¶
+Topic Title
+Subsequent indented lines comprise + the body of the topic, and are + interpreted as body elements.
+1.16 rubric¶
+I like kitty
+1.17 Epigraph¶
+++No matter where you go, there you are.
+—Buckaroo Banzai
+
1.18 compound¶
+The ‘rm’ command is very dangerous. If you are logged + in as root and enter
+cd /
+rm -rf *
+you will erase the entire contents of your file system.
+1.19 Table of Contents¶
+Table of Contents
+-
+
-
+
-
+
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
- +
+
1.21 Awesome¶
+1.22 Replacement Text¶
+Yes, reStructuredText is a long word, so I can’t blame anyone for wanting to + abbreviate it.
+I recommend you try Python, the best language around.
+1.23 Unicode Character Codes¶
+Copyright © 2003, BogusMegaCorp™—all rights reserved.
+1.25 Roles¶
+envvar
… is installed in /usr/lib/python2.x/site-packages …
The function datetime.datetime.today() does a similar thing.
The function datetime.datetime does a similar thing.
The function datetime.INVALID does a similar thing.
1.26 Class description¶
+https://airflow.readthedocs.io/en/latest/_api/airflow/contrib/hooks/gcp_api_base_hook/index.html
+-
+
-
+ class
GoogleCloudBaseHookDDD(gcp_conn_id:str='google_cloud_default', delegate_to:str=None)¶
+ Bases:
+airflow.hooks.base_hook.BaseHookA base hook for Google cloud-related hooks. Google cloud has a shared REST + API client that is built in the same way no matter which service you use. + This class helps construct and authorize the credentials needed to then + call googleapiclient.discovery.build() to actually discover and build a client + for a Google cloud service.
+The class also contains some miscellaneous helper functions.
+All hook derived from this base hook use the ‘Google Cloud Platform’ connection + type. Three ways of authentication are supported:
+Default credentials: Only the ‘Project Id’ is required. You’ll need to + have set up default credentials, such as by the +
+GOOGLE_APPLICATION_DEFAULTenvironment variable or from the metadata + server on Google Compute Engine.JSON key file: Specify ‘Project Id’, ‘Keyfile Path’ and ‘Scope’.
+Legacy P12 key files are not supported.
+JSON data provided in the UI: Specify ‘Keyfile JSON’.
+-
+
- Parameters +
- + +
-
+
-
+
send_message(sender, recipient, message_body[, priority=1])¶
+ Send a message to a recipient
+-
+
- Parameters +
- + +
- Returns +
the message id
+
+ - Return type +
- + +
- Raises +
-
+
ValueError – if the message_body exceeds 160 characters
+ TypeError – if the message_body is not a basestring
+
+
-
+
-
+ static
send_message()
+ Send a message to a recipient
+
-
+
-
+ static
send_message()
+ Send a message to a recipient
+
-
+
-
+ classmethod
send_message()
+ Send a message to a recipient
+
-
+
-
+
@send_message
+ Send a message to a recipient
+
-
+
-
+
@send_message
+ Send a message to a recipient
+
-
+
-
+
num_retries¶
+ Returns num_retries from Connection.
+-
+
- Returns +
the number of times each API request should be retried
+
+ - Return type +
- + +
-
+
-
+
client_info¶
+ Return client information used to generate a user-agent for API calls.
+It allows for better errors tracking.
+This object is only used by the google-cloud-* libraries that are built specifically for + the Google Cloud Platform. It is not supported by The Google APIs Python Client that use Discovery + based APIs.
+
-
+
-
+
scopes¶
+ Return OAuth 2.0 scopes.
+-
+
- Returns +
Returns the scope defined in the connection configuration, or the default scope
+
+ - Return type +
Sequence[str]
+
+
-
+
-
+
_get_credentials_and_project_id(self)¶
+ Returns the Credentials object for Google API and the associated project_id
+
-
+
-
+
_get_credentials(self)¶
+ Returns the Credentials object for Google API
+
-
+
-
+
_get_access_token(self)¶
+ Returns a valid access token from Google API Credentials
+
-
+
+
Returns an authorized HTTP object to be used to build a Google cloud + service hook connection.
+
-
+
-
+
_get_field(self, f:str, default:Any=None)¶
+ Fetches a field from extras, and returns it. This is some Airflow + magic. The google_cloud_platform hook type adds custom UI elements + to the hook page, which allow admins to specify service_account, + key_path, etc. They get formatted as shown below.
+
-
+
-
+ static
catch_http_exception(func:Callable[..., RT])¶
+ Function decorator that intercepts HTTP Errors and raises AirflowException + with more informative message.
+
-
+
-
+ static
fallback_to_default_project_id(func:Callable[..., RT])¶
+ Decorator that provides fallback for Google Cloud Platform project id. If + the project is None it will be replaced with the project_id from the + service account the Hook is authenticated with. Project id can be specified + either via project_id kwarg or via first parameter in positional args.
+-
+
- Parameters +
func – function to wrap
+
+ - Returns +
result of the function call
+
+
-
+
-
+ static
provide_gcp_credential_file(func:Callable[..., RT])¶
+ Function decorator that provides a
+GOOGLE_APPLICATION_CREDENTIALS+ environment variable, pointing to file path of a JSON file of service + account key.
1.27 exampleinclude¶
+
+# In the case of Airflow, the customer needs to create a transfer
+# config with the automatic scheduling disabled and then trigger
+# a transfer run using a specialized Airflow operator
+schedule_options = {"disable_auto_scheduling": True}
+
+PARAMS = {
+ "field_delimiter": ",",
+ "max_bad_records": "0",
+ "skip_leading_rows": "1",
+ "data_path_template": BUCKET_URI,
+ "destination_table_name_template": GCP_DTS_BQ_TABLE,
+ "file_format": "CSV",
+}
+
+TRANSFER_CONFIG = ParseDict(
+ {
+ "destination_dataset_id": GCP_DTS_BQ_DATASET,
+ "display_name": "GCS Test Config",
+ "data_source_id": "google_cloud_storage",
+ "schedule_options": schedule_options,
+ "params": PARAMS,
+ },
+ TransferConfig(),
+)
+
+