Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Managed HTTP Request Support in provider-http #7

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
db95d6c
# Initial commit for expanding provider-http to support managed HTTP …
arielsepton Nov 6, 2023
1791e12
add request resource sample
arielsepton Nov 6, 2023
b348f02
add a base for request controller
arielsepton Nov 6, 2023
0ae53ed
add a base for observe function
arielsepton Nov 6, 2023
f81ebac
add processing for mapping using gojq
arielsepton Nov 6, 2023
5090d1b
add desired state validation in observe function
arielsepton Nov 6, 2023
794eb9b
add handling for non existing resource
arielsepton Nov 6, 2023
16165e0
add update conditions in observe func
arielsepton Nov 6, 2023
950cd75
make 'set_status.go more' flexible
arielsepton Nov 7, 2023
d10f72f
remove func handleDeployActionError and instead reuse status setters
arielsepton Nov 7, 2023
0299234
remove code duplication of IsRequestValid functions
arielsepton Nov 7, 2023
aa7795e
remove code duplication of shouldRetry and waitTimeout functions
arielsepton Nov 7, 2023
c2a25ff
separate to packages
arielsepton Nov 7, 2023
30cb269
remove TODO comment
arielsepton Nov 7, 2023
5be5fea
implement all put and delete
arielsepton Nov 7, 2023
74dd934
remove redundent variable
arielsepton Nov 7, 2023
276555d
change mapping to jq query string instead of json
arielsepton Nov 12, 2023
d84aafb
add json utils package
arielsepton Nov 12, 2023
8490fa4
modify package name and add generateHeaders func
arielsepton Nov 12, 2023
2fc9c09
beautify code and add loggers
arielsepton Nov 12, 2023
21f5431
modify file name and add ApplyJQOnMap func for headers
arielsepton Nov 12, 2023
cadf6dc
remove comment
arielsepton Nov 12, 2023
83a3052
modify file name from requestmapping to requestprocessing
arielsepton Nov 12, 2023
7285f29
add ParseMapStrings func
arielsepton Nov 12, 2023
0fbe411
add ConvertJSONStringsToMaps func
arielsepton Nov 12, 2023
0001d75
use generated headers in request.go
arielsepton Nov 12, 2023
db7670f
remove comment
arielsepton Nov 15, 2023
11e6f03
add validation on mapping field
arielsepton Nov 15, 2023
4da4672
create ParseMapStrings func and add comments
arielsepton Nov 16, 2023
676532c
support only GET POST PUT DELETE mappings
arielsepton Nov 16, 2023
5b2e05e
add comments, beautify
arielsepton Nov 16, 2023
c6c8d5d
remove redundent fields
arielsepton Nov 16, 2023
6cd8f3d
remove redundent fields
arielsepton Nov 16, 2023
748a202
improve examples
arielsepton Nov 16, 2023
0408f78
add docs and improve readme
arielsepton Nov 16, 2023
62fb329
modify example in docs
arielsepton Nov 16, 2023
8df8b5f
Update desposiblerequest_docs.md
arielsepton Nov 16, 2023
c438265
modify package name from httprequest to utils
arielsepton Nov 16, 2023
1689fcc
modify package name from httprequest to utils
arielsepton Nov 16, 2023
3f51904
add method to response to detect if POST failed
arielsepton Nov 16, 2023
4260dac
handle requests failures better
arielsepton Nov 16, 2023
83831a8
remove redundent log
arielsepton Nov 17, 2023
79e91ce
add jq query docs
arielsepton Nov 17, 2023
d1d69c8
add Cache support
arielsepton Nov 19, 2023
ee2ac15
add cache to request CRD
arielsepton Nov 19, 2023
fb5e9db
cache handling in request.go and modify utils
arielsepton Nov 19, 2023
b5b2b5b
add comments for functions
arielsepton Nov 19, 2023
ff21d43
modify function
arielsepton Nov 19, 2023
279c8ea
fix isJsonString func
arielsepton Nov 20, 2023
a4e765d
handle non json responses
arielsepton Nov 20, 2023
139ea4d
improve shouldSetCache
arielsepton Nov 20, 2023
34d5789
fix errors
arielsepton Nov 20, 2023
211f795
add TODO
arielsepton Nov 20, 2023
98a010b
remove redundent condition
arielsepton Nov 22, 2023
cd4ce59
add statushandler for Request
arielsepton Nov 22, 2023
5134100
reset failures reset's error as well
arielsepton Nov 24, 2023
6da9483
modify request's status handler
arielsepton Nov 24, 2023
72db2f8
remove redundent retries check
arielsepton Nov 24, 2023
563c738
remove redundent retries check
arielsepton Nov 24, 2023
92fa013
improve statusHandler
arielsepton Nov 26, 2023
7fc0db4
add unit tests and minor bug fixes
arielsepton Dec 8, 2023
70e5cd6
add unit tests and minor bug fixes
arielsepton Dec 10, 2023
4f684ee
tests and minor code improvements
arielsepton Dec 10, 2023
7f85e10
Get the latest version of the resource before updating
arielsepton Dec 13, 2023
f79674d
add skipTLSVerify
arielsepton Dec 13, 2023
fce4cc2
Get the latest version of the resource before updating
arielsepton Dec 13, 2023
be00131
improve examples
arielsepton Dec 13, 2023
3509a72
beautify client
arielsepton Dec 13, 2023
f552287
update makefile and README
arielsepton Dec 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,8 @@ CROSSPLANE_NAMESPACE = crossplane-system
-include build/makelib/local.xpkg.mk
-include build/makelib/controlplane.mk

UPTEST_EXAMPLE_LIST ?= "examples/sample/desposiblerequest.yaml"
UPTEST_EXAMPLE_LIST := $(shell find ./examples/sample -path '*.yaml' | paste -s -d ',' - )

uptest: $(UPTEST) $(KUBECTL) $(KUTTL)
@$(INFO) running automated tests
@KUBECTL=$(KUBECTL) KUTTL=$(KUTTL) $(UPTEST) e2e "$(UPTEST_EXAMPLE_LIST)" --setup-script=cluster/test/setup.sh || $(FAIL)
Expand Down
72 changes: 54 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,76 @@
# provider-http

`provider-http` is a Crossplane Provider that enables sending one-time HTTP requests as disposable resources.
`provider-http` is a Crossplane Provider designed to facilitate sending HTTP requests as resources.


## Installation

If you would like to install `provider-http` without modifications, you can use the Crossplane CLI in a Kubernetes cluster where Crossplane is installed:
To install `provider-http`, you have two options:

```console
kubectl crossplane install provider crossplanecontrib/provider-http:master
```
1. Using the Crossplane CLI in a Kubernetes cluster where Crossplane is installed:

```console
kubectl crossplane install provider ghcr.io/arielsepton/provider-http:v1.2.0
```

2. Manually creating a Provider by applying the following YAML:

```yaml
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-http
spec:
package: "ghcr.io/arielsepton/provider-http:v1.2.0"
```


## Supported Resources

`provider-http` supports the following resources:

- **DisposableRequest:** Initiates a one-time HTTP request. See [DisposableRequest CRD documentation](resources-docs/desposiblerequest_docs.md).
- **Request:** Manages a resource through HTTP requests. See [Request CRD documentation](resources-docs/request_docs.md).

## Usage

You can also manually install `provider-http` by creating a Provider directly:
### DisposableRequest

Create a `DisposableRequest` resource to initiate a single-use HTTP interaction:

```yaml
apiVersion: pkg.crossplane.io/v1
kind: Provider
apiVersion: http.crossplane.io/v1alpha1
kind: DisposableRequest
metadata:
name: provider-http
name: example-disposable-request
spec:
package: "crossplanecontrib/provider-http:master"
# Add your DisposableRequest specification here
```
For more detailed examples and configuration options, refer to the [examples directory](examples/sample/).

## Usage

Currently, `provider-http` supports only one-time requests using the `DisposableRequest` Custom Resource Definition (CRD). This enables you to initiate a single-use HTTP interaction by creating a `DisposableRequest` resource.
### Request

## Developing locally
Manage a resource through HTTP requests with a `Request` resource:

Run controller against the cluster:
```yaml
apiVersion: http.crossplane.io/v1alpha1
kind: Request
metadata:
name: example-request
spec:
# Add your Request specification here
```
For more detailed examples and configuration options, refer to the [examples directory](examples/sample/).


### Developing locally

Run controller against the cluster:
```
make run
```

Now you can create `DesposibleRequest` resources with provider reference, see [sample desposiblerequest.yaml](examples/sample/desposiblerequest.yaml).

```
kubectl create -f examples/sample/desposiblerequest.yaml
```
### Troubleshooting
If you encounter any issues during installation or usage, refer to the [troubleshooting guide](https://docs.crossplane.io/knowledge-base/guides/troubleshoot/) for common problems and solutions.
32 changes: 16 additions & 16 deletions apis/desposiblerequest/v1alpha1/desposiblerequest_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,30 +27,31 @@ import (

// DesposibleRequestParameters are the configurable fields of a DesposibleRequest.
type DesposibleRequestParameters struct {
URL string `json:"url"`
Method string `json:"method"`
Headers map[string][]string `json:"headers,omitempty"`
Body string `json:"body,omitempty"`
WaitTimeout *metav1.Duration `json:"waitTimeout,omitempty"`
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Field 'forProvider.url' is immutable"
URL string `json:"url"`
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Field 'forProvider.method' is immutable"
Method string `json:"method"`
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Field 'forProvider.headers' is immutable"
Headers map[string][]string `json:"headers,omitempty"`
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Field 'forProvider.body' is immutable"
Body string `json:"body,omitempty"`

WaitTimeout *metav1.Duration `json:"waitTimeout,omitempty"`

// RollbackRetriesLimit is max number of attempts to retry HTTP request by sending again the request.
RollbackRetriesLimit *int32 `json:"rollbackLimit,omitempty"`
}

// DesposibleRequestObservation are the observable fields of a DesposibleRequest.
type DesposibleRequestObservation struct {
ObservableField string `json:"observableField,omitempty"`
// InsecureSkipTLSVerify, when set to true, skips TLS certificate checks for the HTTP request
InsecureSkipTLSVerify bool `json:"insecureSkipTLSVerify,omitempty"`
}

// A DesposibleRequestSpec defines the desired state of a DesposibleRequest.
type DesposibleRequestSpec struct {
xpv1.ResourceSpec `json:",inline"`

// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Field 'ForProvider' is immutable"
ForProvider DesposibleRequestParameters `json:"forProvider"`
}

// DesposibleRequestObservation are the observable fields of a DesposibleRequest.
type Response struct {
StatusCode int `json:"statusCode,omitempty"`
Body string `json:"body,omitempty"`
Expand All @@ -60,11 +61,10 @@ type Response struct {
// A DesposibleRequestStatus represents the observed state of a DesposibleRequest.
type DesposibleRequestStatus struct {
xpv1.ResourceStatus `json:",inline"`
AtProvider DesposibleRequestObservation `json:"atProvider,omitempty"`
Response Response `json:"response,omitempty"`
Failed int32 `json:"failed,omitempty"`
Error string `json:"error,omitempty"`
Synced bool `json:"synced,omitempty"`
Response Response `json:"response,omitempty"`
Failed int32 `json:"failed,omitempty"`
Error string `json:"error,omitempty"`
Synced bool `json:"synced,omitempty"`
}

// +kubebuilder:object:root=true
Expand Down
25 changes: 25 additions & 0 deletions apis/desposiblerequest/v1alpha1/status_setters.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package v1alpha1

func (d *DesposibleRequest) SetStatusCode(statusCode int) {
d.Status.Response.StatusCode = statusCode
}

func (d *DesposibleRequest) SetHeaders(headers map[string][]string) {
d.Status.Response.Headers = headers
}

func (d *DesposibleRequest) SetBody(body string) {
d.Status.Response.Body = body
}

func (d *DesposibleRequest) SetSynced(synced bool) {
d.Status.Synced = synced
}

func (d *DesposibleRequest) SetError(err error) {
d.Status.Failed++
d.SetSynced(true)
if err != nil {
d.Status.Error = err.Error()
}
}
16 changes: 0 additions & 16 deletions apis/desposiblerequest/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions apis/http.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,17 @@ package apis
import (
"k8s.io/apimachinery/pkg/runtime"

samplev1alpha1 "github.com/arielsepton/provider-http/apis/desposiblerequest/v1alpha1"
desposiblerequestv1alpha1 "github.com/arielsepton/provider-http/apis/desposiblerequest/v1alpha1"
requestv1alpha1 "github.com/arielsepton/provider-http/apis/request/v1alpha1"
httpv1alpha1 "github.com/arielsepton/provider-http/apis/v1alpha1"
)

func init() {
// Register the types with the Scheme so the components can map objects to GroupVersionKinds and back
AddToSchemes = append(AddToSchemes,
httpv1alpha1.SchemeBuilder.AddToScheme,
samplev1alpha1.SchemeBuilder.AddToScheme,
desposiblerequestv1alpha1.SchemeBuilder.AddToScheme,
requestv1alpha1.SchemeBuilder.AddToScheme,
)
}

Expand Down
18 changes: 18 additions & 0 deletions apis/request/request.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*
Copyright 2022 The Crossplane Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// Package request contains group request API versions
package request
17 changes: 17 additions & 0 deletions apis/request/v1alpha1/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/*
Copyright 2022 The Crossplane Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha1
40 changes: 40 additions & 0 deletions apis/request/v1alpha1/groupversion_info.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/*
Copyright 2020 The Crossplane Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// Package v1alpha1 contains the v1alpha1 group Sample resources of the http provider.
// +kubebuilder:object:generate=true
// +groupName=http.crossplane.io
// +versionName=v1alpha1
package v1alpha1

import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/scheme"
)

// Package type metadata.
const (
Group = "http.crossplane.io"
Version = "v1alpha1"
)

var (
// SchemeGroupVersion is group version used to register these objects
SchemeGroupVersion = schema.GroupVersion{Group: Group, Version: Version}

// SchemeBuilder is used to add go types to the GroupVersionKind scheme
SchemeBuilder = &scheme.Builder{GroupVersion: SchemeGroupVersion}
)
Loading
Loading