From cb6ff8948a1569185af39ca5596fa6cac306bafc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 01:23:16 +0000 Subject: [PATCH 001/150] README with table of contents and first indexed file --- README.md | 57 +++++++++++++++++++++++++++++++++++++++-- under-the-hood/index.md | 38 +++++++++++++++++++++++++++ 2 files changed, 93 insertions(+), 2 deletions(-) create mode 100644 under-the-hood/index.md diff --git a/README.md b/README.md index 00876ad..3c6d79c 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,55 @@ -# contexture-docs -Documentation for Contexture +# Contexture Documentation + +This repo is designed to host the documentation of Contexture as a +whole. Everythin we currently have here is **WORK IN PROGRESS**. + +## Table of Contents + +* [About Contexture]() + * [What is Contexture]() + * [Map of Repos]() + * [Brief History]() + * [Alternatives]() +* [Getting Started]() + * [Project Setup & Use]() + * [Connecting to Elasticsearch]() + * [Connecting to Other Databases]() + * [Simple Search Box]() + * [Your First Filter]() + * [Discovering the Database]() + * [IMDB Index]() +* [Querying]() + * Contexture DSL. + * Available providers. + * Available types. +* [Components]() +* [Managing State]() + - MobX + - Redux (Coming Soon) +* [Theming]() +* [Recommendations]() + * [Architecture]() + - Client vs Server + - Type definitions + - Scaling + * [Server Side Searches]() + * [Searching On an Endpoint]() + * [Caching]() + * [Client Side Searches]() + * [Click to Search]() + * [Real Time Searches]() + * [Cascading]() +* [Under the Hood]() + * [Design Principles]() + * Contexture Core + - Contexture initializer + - Providers + - Types + - Reactors + * Contexture ElasticSearch + * Contexture Mongo + * Contexture Client + * Contexture React +* [Examples]() +* [Contributing Guide]() +* [License]() diff --git a/under-the-hood/index.md b/under-the-hood/index.md new file mode 100644 index 0000000..fedf107 --- /dev/null +++ b/under-the-hood/index.md @@ -0,0 +1,38 @@ +# Under The Hood + +## Design Principles + +- Intentionally stateless. +- Detached from state management tools. +- Can work well with state management tools. +- Modern ES6+ +- Non-strict functional programming. +- Configuration based architecture. +- Focus on a very extensible small core. +- Small DSL. + - Database agnostic. + - Isomorphic Tree State. + - Optimized for database discovery. + - Optimized for advanced search interfaces. + - Aiming to be effective for arbitrarily complex database indexes. + - Simplicity over performance. +- Reaction management. + - Tree walking. + - State flags. + - What updates. + - Pauses. + +## Contexture Core + +- Initializer. + - Arguments and return values. + - What it does. + - Initial values in the tree. + - Flat lenses. + - State flags. + - Add filters. + - DFS initialization. + - Example usages. +- Core Utils. + +## Contexture ElasticSearch From 7f1ceb41099c9226801c8767839dae1713dc327d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 16:36:45 +0000 Subject: [PATCH 002/150] current progress --- under-the-hood/contexture-core.md | 40 ++++++++++++++++++++++++++ under-the-hood/contexture-providers.md | 20 +++++++++++++ under-the-hood/design-principles.md | 1 + under-the-hood/index.md | 17 +++++++++-- 4 files changed, 75 insertions(+), 3 deletions(-) create mode 100644 under-the-hood/contexture-core.md create mode 100644 under-the-hood/contexture-providers.md create mode 100644 under-the-hood/design-principles.md diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md new file mode 100644 index 0000000..a0db52c --- /dev/null +++ b/under-the-hood/contexture-core.md @@ -0,0 +1,40 @@ +# Contexture Core + +The core of Contexture is a package of its own. Located at [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture), it offers the very underlying function that is designed to process every one of the search queries. It begins with a simple curried function which, once the providers and the schemas are received, proceeds to process the [Contexture DSL](#TODO) by walking down the given search tree, cleaning up every node of possible inconsistencies, then mutating the tree with the directions given by the provider types and schemas, up until a valid search query is obtained. This query is delivered to the provider `runSearch` method, and the result is finally added back to the tree. + +With this in mind, let's get some specifications. + +## contexture's default export + +Contexture's default export is a curried function with an initial arity of 1, which returns a function with an arity of 2. This separates the initialization from the search execution. + +The first call of Contexture, with an arity 1, expects to receive a plain JavaScript Object with two expected keys: + +- `providers`: It's expected to be an object where each key will have a [Contexture Provider](#TODO). +- `schemas`: It's expected to be an object where each key will have a [Contexture Schema](#TODO). + +Example Declaration: + +```javascript +Contexture({ + schemas: { + ...mongoSchemas, + ...elasticSearchSchemas, + }, + providers: { + mongo: require('contexture-mongo')({ /* provider configuration */ }), + elasticsearch: require('contexture-elasticsearch')({ /* provider configuration */}), + }, +}) +``` + +With that object sent, Contexture is officially initialized 🎉.The +resulting function will be able to be called multiple times to run +searches. This function expects the search tree ([Contexture DSL](#TODO)), and optionally an object that can have the following properties: + +| Option | Description | +| ------ | ----------- | +| `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info | +| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example | + +This second function will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the [Contexture Types](#TODO) diff --git a/under-the-hood/contexture-providers.md b/under-the-hood/contexture-providers.md new file mode 100644 index 0000000..975979f --- /dev/null +++ b/under-the-hood/contexture-providers.md @@ -0,0 +1,20 @@ +# Contexture Providers + +The _Contexture Providers_ are an abstraction layer around the databases that can be targeted by any given search tree. Each Provider by itself is a function that is called for initialization, which expects the following parameters: + +| Input Object Property | Meaning | +| ------- | ------- | +| `types` | [Contexture Types](#TODO) that apply for this specific provider | +| `getClient` | Function that will return the direct database API manager, such as the `mongodb` NPM package. | + +This initialization retutns an object with: + +| Return Object Property | Meaning | +| ------- | ------- | +| `types` | [Contexture Types](#TODO) that apply for this specific provider | +| `groupCombinator` | Function that will determine how to construct boolean operations on the search tree nodes, such as `and`, `or` and `not`. | + +With that in mind, let's explore our currently available providers: + +## contexture-mongo +## contexture-elasticsearch diff --git a/under-the-hood/design-principles.md b/under-the-hood/design-principles.md new file mode 100644 index 0000000..b1c1b6a --- /dev/null +++ b/under-the-hood/design-principles.md @@ -0,0 +1 @@ +# Design Principles diff --git a/under-the-hood/index.md b/under-the-hood/index.md index fedf107..eb63768 100644 --- a/under-the-hood/index.md +++ b/under-the-hood/index.md @@ -22,9 +22,17 @@ - What updates. - Pauses. -## Contexture Core +## DSL + +## Contexture Types +- Group +- Query +- Filters (Terms) +- Result -- Initializer. +## Contexture Core +- Contexture Default Export. + - Initialization Parameters. - Arguments and return values. - What it does. - Initial values in the tree. @@ -35,4 +43,7 @@ - Example usages. - Core Utils. -## Contexture ElasticSearch +## Contexture Providers + - Design Structure + - Contexture ElasticSearch + - Contexture Mongo From b145ed5d38784ef81642fed902d8f61a3bc996ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 17:40:31 +0000 Subject: [PATCH 003/150] Feedback in contexture-core Thanks Sam! --- under-the-hood/contexture-core.md | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md index a0db52c..4c88be4 100644 --- a/under-the-hood/contexture-core.md +++ b/under-the-hood/contexture-core.md @@ -6,17 +6,18 @@ With this in mind, let's get some specifications. ## contexture's default export -Contexture's default export is a curried function with an initial arity of 1, which returns a function with an arity of 2. This separates the initialization from the search execution. +Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. The first argument is expected to be a plain JavaScript Object with two keys: -The first call of Contexture, with an arity 1, expects to receive a plain JavaScript Object with two expected keys: +- `providers`: Should be an object where each key will have a [Contexture Provider](#TODO). +- `schemas`: Should be an object where each key will have a [Contexture Schema](#TODO). -- `providers`: It's expected to be an object where each key will have a [Contexture Provider](#TODO). -- `schemas`: It's expected to be an object where each key will have a [Contexture Schema](#TODO). +Calling this function with this object only will return another function, which can be used as an asynchronous search runner. You can also pass in all the arguments as once, but the separation of parameters makes it easier to scope setting up the database providers, the types and the schemas from the search execution. -Example Declaration: +Example declaration of a search function by passing the schema & +providers object first: ```javascript -Contexture({ +const search = Contexture({ schemas: { ...mongoSchemas, ...elasticSearchSchemas, @@ -26,15 +27,19 @@ Contexture({ elasticsearch: require('contexture-elasticsearch')({ /* provider configuration */}), }, }) + +// How you might use it with an express-like API: +// app.use('/search', async req, res) => +// res.send(await search(req.body.search)) ``` -With that object sent, Contexture is officially initialized 🎉.The -resulting function will be able to be called multiple times to run -searches. This function expects the search tree ([Contexture DSL](#TODO)), and optionally an object that can have the following properties: +The other two parameters are the search tree (the [Contexture +DSL](#TODO)), and an optional object with the following properties: | Option | Description | | ------ | ----------- | -| `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info | -| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example | +| `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info. | +| `onResult` | A callback which is called whenever a node finishes +producing it's results, which can be used to send partial results over websockets for example. | -This second function will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the [Contexture Types](#TODO) +This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the [Contexture Types](#TODO). From 74e6be3191e6368ef5ee05ae4f43a9fb44409d9f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 20:45:47 +0000 Subject: [PATCH 004/150] More content --- README.md | 4 +++- types/diy-types.md | 12 ++++++++++++ under-the-hood/contexture-core.md | 19 ++++++++++++++++--- under-the-hood/contexture-providers.md | 9 ++++++++- under-the-hood/index.md | 15 +-------------- 5 files changed, 40 insertions(+), 19 deletions(-) create mode 100644 types/diy-types.md diff --git a/README.md b/README.md index 3c6d79c..f7858bd 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,9 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * [Querying]() * Contexture DSL. * Available providers. - * Available types. +* [Types]() + * [DIY Types]() + * [Example Types]() * [Components]() * [Managing State]() - MobX diff --git a/types/diy-types.md b/types/diy-types.md new file mode 100644 index 0000000..806494f --- /dev/null +++ b/types/diy-types.md @@ -0,0 +1,12 @@ +# DIY Types + +The Contexture ecosystem provides a defined list of types that can be +used to perform a wide variety of different searches. Our types are +focused on the different search interfaces that we provide, but our +API is designed to allow you to build any type you might need for any +other possible use case you might encounter. We believe in making a +generic framework so you can be creative on your search solutions. +Because of that, we will start this chapter by explaining how to build +your own types. + +### diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md index 4c88be4..8637b00 100644 --- a/under-the-hood/contexture-core.md +++ b/under-the-hood/contexture-core.md @@ -6,15 +6,18 @@ With this in mind, let's get some specifications. ## contexture's default export -Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. The first argument is expected to be a plain JavaScript Object with two keys: +Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. + +### Arguments and Return Values + +The first argument is expected to be a plain JavaScript Object with two keys: - `providers`: Should be an object where each key will have a [Contexture Provider](#TODO). - `schemas`: Should be an object where each key will have a [Contexture Schema](#TODO). Calling this function with this object only will return another function, which can be used as an asynchronous search runner. You can also pass in all the arguments as once, but the separation of parameters makes it easier to scope setting up the database providers, the types and the schemas from the search execution. -Example declaration of a search function by passing the schema & -providers object first: +Example declaration of a search function by passing the schema & providers object first: ```javascript const search = Contexture({ @@ -43,3 +46,13 @@ DSL](#TODO)), and an optional object with the following properties: producing it's results, which can be used to send partial results over websockets for example. | This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the [Contexture Types](#TODO). + +## The Algorithm + +_Coming soon..._ + +- Initial values in the tree. +- Flat lenses. +- State flags. +- Add filters. +- DFS initialization. diff --git a/under-the-hood/contexture-providers.md b/under-the-hood/contexture-providers.md index 975979f..25fb01b 100644 --- a/under-the-hood/contexture-providers.md +++ b/under-the-hood/contexture-providers.md @@ -1,6 +1,6 @@ # Contexture Providers -The _Contexture Providers_ are an abstraction layer around the databases that can be targeted by any given search tree. Each Provider by itself is a function that is called for initialization, which expects the following parameters: +The _Contexture Providers_ are an abstraction layer around the databases that can be targeted by any given search tree. Each Provider is a function that expects the following parameters: | Input Object Property | Meaning | | ------- | ------- | @@ -17,4 +17,11 @@ This initialization retutns an object with: With that in mind, let's explore our currently available providers: ## contexture-mongo +- Where the package lives +- What it does +- list of types (not much detail, link to the details docs) + ## contexture-elasticsearch +- Where the package lives +- What it does +- list of types (not much detail, link to the details docs) diff --git a/under-the-hood/index.md b/under-the-hood/index.md index eb63768..1bd5328 100644 --- a/under-the-hood/index.md +++ b/under-the-hood/index.md @@ -24,23 +24,10 @@ ## DSL -## Contexture Types -- Group -- Query -- Filters (Terms) -- Result - ## Contexture Core - Contexture Default Export. - - Initialization Parameters. - Arguments and return values. - - What it does. - - Initial values in the tree. - - Flat lenses. - - State flags. - - Add filters. - - DFS initialization. - - Example usages. + - The Algorithm. - Core Utils. ## Contexture Providers From bd6d4bba94e1f4d5c8697144eac25222765e73dd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 21:09:58 +0000 Subject: [PATCH 005/150] More content --- README.md | 1 + types/diy-types.md | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f7858bd..3ecfe27 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * Available providers. * [Types]() * [DIY Types]() + * [How to Write a Type]() * [Example Types]() * [Components]() * [Managing State]() diff --git a/types/diy-types.md b/types/diy-types.md index 806494f..4b1c62d 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -9,4 +9,17 @@ generic framework so you can be creative on your search solutions. Because of that, we will start this chapter by explaining how to build your own types. -### +## How to Wite a Type + +Writing a type is as simple as writing a plain JavaScript Object with +one or more of the following properties: + +| Property | Type | Params (Type) | Return Value Type | What it does | +| --- | --- | --- | +| `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | +| `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | +| `result` | Function | Node (Object), Search (Function) | Promise | Allows you to run a direct search for this type before Comtexture sends the full seaech with the whole tree. | + +## Some examples + +_Coming Soon..._ From 85f734fee28a1a2af1a6dba5a763772b7433586d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 21:11:12 +0000 Subject: [PATCH 006/150] Table magic --- types/diy-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/types/diy-types.md b/types/diy-types.md index 4b1c62d..a5d3398 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -15,7 +15,7 @@ Writing a type is as simple as writing a plain JavaScript Object with one or more of the following properties: | Property | Type | Params (Type) | Return Value Type | What it does | -| --- | --- | --- | +| --- | --- | --- | --- | --- | | `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | | `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | | `result` | Function | Node (Object), Search (Function) | Promise | Allows you to run a direct search for this type before Comtexture sends the full seaech with the whole tree. | From 845e7285c28429d7f7f84d91f32e0b4b2f663fce Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 22:23:11 +0000 Subject: [PATCH 007/150] More about types --- README.md | 7 +- .../contexture-elasticsearch-example-types.md | 18 +++++ types/diy-types.md | 76 +++++++++++++++++-- 3 files changed, 93 insertions(+), 8 deletions(-) create mode 100644 types/contexture-elasticsearch-example-types.md diff --git a/README.md b/README.md index 3ecfe27..e94f7c4 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,12 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * [Types]() * [DIY Types]() * [How to Write a Type]() - * [Example Types]() + * [How to Write a UI Component for a Type]() + * [Shared Types]() + * [The Example Types]() + * [ElasticSearch Example Types]() + * [Mongo Example Types]() + * [Available React Components for our Types]() * [Components]() * [Managing State]() - MobX diff --git a/types/contexture-elasticsearch-example-types.md b/types/contexture-elasticsearch-example-types.md new file mode 100644 index 0000000..0fb7888 --- /dev/null +++ b/types/contexture-elasticsearch-example-types.md @@ -0,0 +1,18 @@ +# ElasticSearch Example Types + +Contexture is designed to target any database you might need. However, +so far we have only inplemented database providers for the only +databases that we use: ElasticSearch and Mongo. + +Most of our types have relevant components already written to +facilitate building search interfaces. As we progress over each one of +these types, we will show small examples of simple components written +to search with these types. We hope to provide enough information to +allow you to take as little as you need, or as much as you want, and +fulfill you expectations. + +Our ElasticSearch types are the following ones: + +## [bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js) + +The bool type is intended to work as an ElasticSearch terms aggregation with only one value for a single property. This is useful for user interfaces with a checkbox to include or exclude a specific field from a search (or a specific field-value pair). diff --git a/types/diy-types.md b/types/diy-types.md index a5d3398..bc3bcff 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -1,8 +1,10 @@ # DIY Types The Contexture ecosystem provides a defined list of types that can be -used to perform a wide variety of different searches. Our types are -focused on the different search interfaces that we provide, but our +used to perform a wide variety of different searches. Each type we +offer also has a respective component in our contexture-react repo, so +you can quickstart your search interfaces. However, even if our types are +focused on the different search interfaces that we provide, our API is designed to allow you to build any type you might need for any other possible use case you might encounter. We believe in making a generic framework so you can be creative on your search solutions. @@ -11,15 +13,75 @@ your own types. ## How to Wite a Type -Writing a type is as simple as writing a plain JavaScript Object with -one or more of the following properties: +Writing a type is as simple as exposing any valid string property name +on the types object of a specific provider, then assigning a plain +JavaScript Object value with one or more of the following properties: | Property | Type | Params (Type) | Return Value Type | What it does | | --- | --- | --- | --- | --- | -| `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | +| `hasValue` | Function | NodeSome Examples Object) | Boolean | Allows Contexture to know wether or not to process this search. | | `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | | `result` | Function | Node (Object), Search (Function) | Promise | Allows you to run a direct search for this type before Comtexture sends the full seaech with the whole tree. | -## Some examples +Once you have your type written, you can use it by sending it to an +existing [Contexture Provider](#TODO). It should look more or less +like: -_Coming Soon..._ +```javascript +let myType = { + hasValue: node => node.requiredProperty, + filter: node => ({ + providerQueryObject: { + value: node.requiredProperty + } + }) +} + +let provider = MyProvider({ + types: { + myType + } +}) +``` + +## How to Write a UI Component for a Type + +Writing a ussr interface for any type can be as simple as writing an +HTML or JSX Element that will render or write into any property of the +type, for example, using our custom type `myType`, we could write an +input field that, onChange, will write the field's value into the +`requiredProperty`. For example: + +```javascript +let Component = node => ( + { + node.requiredProperty = e.target.value + }} + /> +) +``` + +- Managing State + +## Shared Types + +**Type names are not exclusive across providers**. You can define one +type called `myAwesomeType` in more than one provider and you'll be +able to keep the same required node properties, thus the same +`hasValue`. This allows us to provide the same API for several types, +and re-use code even if we switch the target database of the search. + +## The Example Types + +With the intention of providing practical examples of how to write +types, we decided to share some of the types we use in our production +applications. These types belong to two different database processors: +`contexture-elasticsearch` and `contexture-mongo`. + +**Example Types aren't the rule**. These types are only provided to +serve as a guide to build any other type you might need for your +application. You can also **extend our example types**, simply by +assigning new types as properties on the objects exposed by +`contexture-elasticsearch` and `contexture-mongo`. From d7a5ddf305b5c9977349d65e746a16899676b642 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 23:39:58 +0000 Subject: [PATCH 008/150] More on types --- README.md | 26 ++++---- .../contexture-elasticsearch-example-types.md | 11 +++- types/{diy-types.md => index.md} | 59 ++++++++++++++++--- 3 files changed, 76 insertions(+), 20 deletions(-) rename types/{diy-types.md => index.md} (65%) diff --git a/README.md b/README.md index e94f7c4..987417a 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * [What is Contexture]() * [Map of Repos]() * [Brief History]() - * [Alternatives]() + * [Alternatives & Benchmarks]() * [Getting Started]() * [Project Setup & Use]() * [Connecting to Elasticsearch]() @@ -19,9 +19,13 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * [Discovering the Database]() * [IMDB Index]() * [Querying]() + * Contexture Core. * Contexture DSL. * Available providers. -* [Types]() +* [Interactive Queries]() + * Contexture Client. + * Introduction to Reactors. +* [Types and Type Components]() * [DIY Types]() * [How to Write a Type]() * [How to Write a UI Component for a Type]() @@ -30,16 +34,16 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * [ElasticSearch Example Types]() * [Mongo Example Types]() * [Available React Components for our Types]() -* [Components]() +* [Other Components]() * [Managing State]() - MobX - Redux (Coming Soon) * [Theming]() * [Recommendations]() * [Architecture]() - - Client vs Server - - Type definitions - - Scaling + - Client vs Server. + - Type definitions. + - Scaling. * [Server Side Searches]() * [Searching On an Endpoint]() * [Caching]() @@ -49,11 +53,11 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. * [Cascading]() * [Under the Hood]() * [Design Principles]() - * Contexture Core - - Contexture initializer - - Providers - - Types - - Reactors + * Contexture Core. + - Contexture initializer. + - Providers. + - Types. + - Reactors in detail. * Contexture ElasticSearch * Contexture Mongo * Contexture Client diff --git a/types/contexture-elasticsearch-example-types.md b/types/contexture-elasticsearch-example-types.md index 0fb7888..2afe3be 100644 --- a/types/contexture-elasticsearch-example-types.md +++ b/types/contexture-elasticsearch-example-types.md @@ -15,4 +15,13 @@ Our ElasticSearch types are the following ones: ## [bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js) -The bool type is intended to work as an ElasticSearch terms aggregation with only one value for a single property. This is useful for user interfaces with a checkbox to include or exclude a specific field from a search (or a specific field-value pair). +The bool type is intended to work as an ElasticSearch terms +aggregation with only one value for a single property. This is useful +for user interfaces with a checkbox to include or exclude a specific +field from a search (or a specific field-value pair). + +Here's the definition of this type: + +| Property Name | Type | Description | +| --- | --- | --- | +| field | String | Name of the field that we will include or exclude from the search. | diff --git a/types/diy-types.md b/types/index.md similarity index 65% rename from types/diy-types.md rename to types/index.md index bc3bcff..a1ece40 100644 --- a/types/diy-types.md +++ b/types/index.md @@ -13,6 +13,8 @@ your own types. ## How to Wite a Type +### Provider Type + Writing a type is as simple as exposing any valid string property name on the types object of a specific provider, then assigning a plain JavaScript Object value with one or more of the following properties: @@ -44,6 +46,55 @@ let provider = MyProvider({ }) ``` +**Type names are not exclusive across providers**. You can define one +type called `myAwesomeType` in more than one provider and you'll be +able to keep the same required node properties, thus the same +`hasValue`. This allows us to provide the same API for several types, +and re-use code even if we switch the target database of the search. + +Once you have a provider type defined for one or more providers, you +should write the same type for `contexture-client`. + +### Contexture Client Type + +Contexture Client already provides a bunch of types based on our +`Example Types` (more on that later.) These type definitions help +the client understand how a specific node affects every other node or +itself. + +To create a custom type, you will need to think on the behaviors you +might need for each one of the following properties: + +| Property Name | Type | Description | +| --- | --- | --- | +| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | +| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | +| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | + +The example types are already included in any instantiation +of Contexture Client's Contexture Tree. However, you can extend them +with any type you need simply by complementing the exposed +`exampleTypes` with your own. Here's an example where we initialize +a `ContextureTree` with the available `exampleTypes`, and our new `myType`: + +```javascript +import * as ContextureClient from 'contexture-client' +let tree = ContextureTree({ + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } +}) +``` + ## How to Write a UI Component for a Type Writing a ussr interface for any type can be as simple as writing an @@ -65,14 +116,6 @@ let Component = node => ( - Managing State -## Shared Types - -**Type names are not exclusive across providers**. You can define one -type called `myAwesomeType` in more than one provider and you'll be -able to keep the same required node properties, thus the same -`hasValue`. This allows us to provide the same API for several types, -and re-use code even if we switch the target database of the search. - ## The Example Types With the intention of providing practical examples of how to write From 50b84713966e688a43601f4d1fb4427aa0dfdd38 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 23:42:12 +0000 Subject: [PATCH 009/150] What I just did on github that I will overwrite with a force push --- types/index.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/types/index.md b/types/index.md index a1ece40..51b9987 100644 --- a/types/index.md +++ b/types/index.md @@ -2,14 +2,16 @@ The Contexture ecosystem provides a defined list of types that can be used to perform a wide variety of different searches. Each type we -offer also has a respective component in our contexture-react repo, so -you can quickstart your search interfaces. However, even if our types are -focused on the different search interfaces that we provide, our -API is designed to allow you to build any type you might need for any -other possible use case you might encounter. We believe in making a -generic framework so you can be creative on your search solutions. -Because of that, we will start this chapter by explaining how to build -your own types. +offer also has a respective component in our `contexture-react` repo. +We've made these components so you can quickstart your search interfaces! + +However, even if our types are focused on the different search interfaces +we provide, our API is designed to allow you to build any type you might need +for any other possible use case you might encounter. + +We believe in making a generic framework so you can be creative on your +search solutions. Because of that, we will start this document by explaining +how to build your own types. ## How to Wite a Type From 434bec80d2a42ad7c53449ca11b740c16c7fb63f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 23:46:43 +0000 Subject: [PATCH 010/150] Playing with links --- README.md | 101 ++++++++++++++++++++++++------------------------- types/index.md | 2 +- 2 files changed, 51 insertions(+), 52 deletions(-) diff --git a/README.md b/README.md index 987417a..52194aa 100644 --- a/README.md +++ b/README.md @@ -5,63 +5,62 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. ## Table of Contents -* [About Contexture]() - * [What is Contexture]() - * [Map of Repos]() - * [Brief History]() - * [Alternatives & Benchmarks]() -* [Getting Started]() - * [Project Setup & Use]() - * [Connecting to Elasticsearch]() - * [Connecting to Other Databases]() - * [Simple Search Box]() - * [Your First Filter]() - * [Discovering the Database]() - * [IMDB Index]() -* [Querying]() - * Contexture Core. - * Contexture DSL. - * Available providers. -* [Interactive Queries]() - * Contexture Client. - * Introduction to Reactors. -* [Types and Type Components]() - * [DIY Types]() - * [How to Write a Type]() - * [How to Write a UI Component for a Type]() - * [Shared Types]() - * [The Example Types]() - * [ElasticSearch Example Types]() - * [Mongo Example Types]() - * [Available React Components for our Types]() -* [Other Components]() -* [Managing State]() +- [About Contexture]() + - [What is Contexture]() + - [Map of Repos]() + - [Brief History]() + - [Alternatives & Benchmarks]() +- [Getting Started]() + - [Project Setup & Use]() + - [Connecting to Elasticsearch]() + - [Connecting to Other Databases]() + - [Simple Search Box]() + - [Your First Filter]() + - [Discovering the Database]() + - [IMDB Index]() +- [Querying]() + - Contexture Core. + - Contexture DSL. + - Available providers. +- [Interactive Queries]() + - Contexture Client. + - Introduction to Reactors. +- **Types and Type Components** + - [DIY Types](types/index.md) + - [How to Write a Type](types/index.md#how-to-wite-a-type) + - [How to Write a UI Component for a Type](types/index.md#how-to-write-a-ui-component-for-a-type) + - [The Example Types](types/index.md#the-example-types) + - [ElasticSearch Example Types]() + - [Mongo Example Types]() + - [Available React Components for our Types]() +- [Other Components]() +- [Managing State]() - MobX - Redux (Coming Soon) -* [Theming]() -* [Recommendations]() - * [Architecture]() +- [Theming]() +- [Recommendations]() + - [Architecture]() - Client vs Server. - Type definitions. - Scaling. - * [Server Side Searches]() - * [Searching On an Endpoint]() - * [Caching]() - * [Client Side Searches]() - * [Click to Search]() - * [Real Time Searches]() - * [Cascading]() -* [Under the Hood]() - * [Design Principles]() - * Contexture Core. + - [Server Side Searches]() + - [Searching On an Endpoint]() + - [Caching]() + - [Client Side Searches]() + - [Click to Search]() + - [Real Time Searches]() + - [Cascading]() +- [Under the Hood]() + - [Design Principles]() + - Contexture Core. - Contexture initializer. - Providers. - Types. - Reactors in detail. - * Contexture ElasticSearch - * Contexture Mongo - * Contexture Client - * Contexture React -* [Examples]() -* [Contributing Guide]() -* [License]() + - Contexture ElasticSearch + - Contexture Mongo + - Contexture Client + - Contexture React +- [Examples]() +- [Contributing Guide]() +- [License]() diff --git a/types/index.md b/types/index.md index 51b9987..f2ccea1 100644 --- a/types/index.md +++ b/types/index.md @@ -1,4 +1,4 @@ -# DIY Types +# Types and Type Components The Contexture ecosystem provides a defined list of types that can be used to perform a wide variety of different searches. Each type we From e09a0d9b4bc1c916bafeb4eaea04e14a2b2ac716 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 25 May 2018 23:53:22 +0000 Subject: [PATCH 011/150] More stuff --- README.md | 10 ++-- types/diy-types.md | 132 ++++++++++++++++++++++++++++++++++++++++++++ types/index.md | 135 ++------------------------------------------- 3 files changed, 142 insertions(+), 135 deletions(-) create mode 100644 types/diy-types.md diff --git a/README.md b/README.md index 52194aa..22ea070 100644 --- a/README.md +++ b/README.md @@ -25,11 +25,11 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Interactive Queries]() - Contexture Client. - Introduction to Reactors. -- **Types and Type Components** - - [DIY Types](types/index.md) - - [How to Write a Type](types/index.md#how-to-wite-a-type) - - [How to Write a UI Component for a Type](types/index.md#how-to-write-a-ui-component-for-a-type) - - [The Example Types](types/index.md#the-example-types) +- [Types and Type Components](types/index.md) + - [DIY Types](types/diy-types.md) + - [How to Write a Type](types/diy-types.md#how-to-wite-a-type) + - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) + - [The Example Types](types/diy-types.md#the-example-types) - [ElasticSearch Example Types]() - [Mongo Example Types]() - [Available React Components for our Types]() diff --git a/types/diy-types.md b/types/diy-types.md new file mode 100644 index 0000000..51b9987 --- /dev/null +++ b/types/diy-types.md @@ -0,0 +1,132 @@ +# DIY Types + +The Contexture ecosystem provides a defined list of types that can be +used to perform a wide variety of different searches. Each type we +offer also has a respective component in our `contexture-react` repo. +We've made these components so you can quickstart your search interfaces! + +However, even if our types are focused on the different search interfaces +we provide, our API is designed to allow you to build any type you might need +for any other possible use case you might encounter. + +We believe in making a generic framework so you can be creative on your +search solutions. Because of that, we will start this document by explaining +how to build your own types. + +## How to Wite a Type + +### Provider Type + +Writing a type is as simple as exposing any valid string property name +on the types object of a specific provider, then assigning a plain +JavaScript Object value with one or more of the following properties: + +| Property | Type | Params (Type) | Return Value Type | What it does | +| --- | --- | --- | --- | --- | +| `hasValue` | Function | NodeSome Examples Object) | Boolean | Allows Contexture to know wether or not to process this search. | +| `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | +| `result` | Function | Node (Object), Search (Function) | Promise | Allows you to run a direct search for this type before Comtexture sends the full seaech with the whole tree. | + +Once you have your type written, you can use it by sending it to an +existing [Contexture Provider](#TODO). It should look more or less +like: + +```javascript +let myType = { + hasValue: node => node.requiredProperty, + filter: node => ({ + providerQueryObject: { + value: node.requiredProperty + } + }) +} + +let provider = MyProvider({ + types: { + myType + } +}) +``` + +**Type names are not exclusive across providers**. You can define one +type called `myAwesomeType` in more than one provider and you'll be +able to keep the same required node properties, thus the same +`hasValue`. This allows us to provide the same API for several types, +and re-use code even if we switch the target database of the search. + +Once you have a provider type defined for one or more providers, you +should write the same type for `contexture-client`. + +### Contexture Client Type + +Contexture Client already provides a bunch of types based on our +`Example Types` (more on that later.) These type definitions help +the client understand how a specific node affects every other node or +itself. + +To create a custom type, you will need to think on the behaviors you +might need for each one of the following properties: + +| Property Name | Type | Description | +| --- | --- | --- | +| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | +| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | +| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | + +The example types are already included in any instantiation +of Contexture Client's Contexture Tree. However, you can extend them +with any type you need simply by complementing the exposed +`exampleTypes` with your own. Here's an example where we initialize +a `ContextureTree` with the available `exampleTypes`, and our new `myType`: + +```javascript +import * as ContextureClient from 'contexture-client' +let tree = ContextureTree({ + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } +}) +``` + +## How to Write a UI Component for a Type + +Writing a ussr interface for any type can be as simple as writing an +HTML or JSX Element that will render or write into any property of the +type, for example, using our custom type `myType`, we could write an +input field that, onChange, will write the field's value into the +`requiredProperty`. For example: + +```javascript +let Component = node => ( + { + node.requiredProperty = e.target.value + }} + /> +) +``` + +- Managing State + +## The Example Types + +With the intention of providing practical examples of how to write +types, we decided to share some of the types we use in our production +applications. These types belong to two different database processors: +`contexture-elasticsearch` and `contexture-mongo`. + +**Example Types aren't the rule**. These types are only provided to +serve as a guide to build any other type you might need for your +application. You can also **extend our example types**, simply by +assigning new types as properties on the objects exposed by +`contexture-elasticsearch` and `contexture-mongo`. diff --git a/types/index.md b/types/index.md index f2ccea1..f8ff6eb 100644 --- a/types/index.md +++ b/types/index.md @@ -1,132 +1,7 @@ # Types and Type Components -The Contexture ecosystem provides a defined list of types that can be -used to perform a wide variety of different searches. Each type we -offer also has a respective component in our `contexture-react` repo. -We've made these components so you can quickstart your search interfaces! - -However, even if our types are focused on the different search interfaces -we provide, our API is designed to allow you to build any type you might need -for any other possible use case you might encounter. - -We believe in making a generic framework so you can be creative on your -search solutions. Because of that, we will start this document by explaining -how to build your own types. - -## How to Wite a Type - -### Provider Type - -Writing a type is as simple as exposing any valid string property name -on the types object of a specific provider, then assigning a plain -JavaScript Object value with one or more of the following properties: - -| Property | Type | Params (Type) | Return Value Type | What it does | -| --- | --- | --- | --- | --- | -| `hasValue` | Function | NodeSome Examples Object) | Boolean | Allows Contexture to know wether or not to process this search. | -| `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | -| `result` | Function | Node (Object), Search (Function) | Promise | Allows you to run a direct search for this type before Comtexture sends the full seaech with the whole tree. | - -Once you have your type written, you can use it by sending it to an -existing [Contexture Provider](#TODO). It should look more or less -like: - -```javascript -let myType = { - hasValue: node => node.requiredProperty, - filter: node => ({ - providerQueryObject: { - value: node.requiredProperty - } - }) -} - -let provider = MyProvider({ - types: { - myType - } -}) -``` - -**Type names are not exclusive across providers**. You can define one -type called `myAwesomeType` in more than one provider and you'll be -able to keep the same required node properties, thus the same -`hasValue`. This allows us to provide the same API for several types, -and re-use code even if we switch the target database of the search. - -Once you have a provider type defined for one or more providers, you -should write the same type for `contexture-client`. - -### Contexture Client Type - -Contexture Client already provides a bunch of types based on our -`Example Types` (more on that later.) These type definitions help -the client understand how a specific node affects every other node or -itself. - -To create a custom type, you will need to think on the behaviors you -might need for each one of the following properties: - -| Property Name | Type | Description | -| --- | --- | --- | -| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | -| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | -| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | - -The example types are already included in any instantiation -of Contexture Client's Contexture Tree. However, you can extend them -with any type you need simply by complementing the exposed -`exampleTypes` with your own. Here's an example where we initialize -a `ContextureTree` with the available `exampleTypes`, and our new `myType`: - -```javascript -import * as ContextureClient from 'contexture-client' -let tree = ContextureTree({ - types: { - ...ContextureClient.exampleTypes, - myType: { - validate: node => node.requiredProperty, - reactors: { - requiredProperty: 'others', - }, - defaults: { - requiredProperty: false - } - } - } -}) -``` - -## How to Write a UI Component for a Type - -Writing a ussr interface for any type can be as simple as writing an -HTML or JSX Element that will render or write into any property of the -type, for example, using our custom type `myType`, we could write an -input field that, onChange, will write the field's value into the -`requiredProperty`. For example: - -```javascript -let Component = node => ( - { - node.requiredProperty = e.target.value - }} - /> -) -``` - -- Managing State - -## The Example Types - -With the intention of providing practical examples of how to write -types, we decided to share some of the types we use in our production -applications. These types belong to two different database processors: -`contexture-elasticsearch` and `contexture-mongo`. - -**Example Types aren't the rule**. These types are only provided to -serve as a guide to build any other type you might need for your -application. You can also **extend our example types**, simply by -assigning new types as properties on the objects exposed by -`contexture-elasticsearch` and `contexture-mongo`. +Table of Contents: +- [DIY Types](diy-types.md) + - [How to Write a Type](diy-types.md#how-to-wite-a-type) + - [How to Write a UI Component for a Type](diy-types.md#how-to-write-a-ui-component-for-a-type) + - [The Example Types](diy-types.md#the-example-types) From e5c55e41f7fddf5222a29edf099f6a51bc75c2e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 01:30:38 +0000 Subject: [PATCH 012/150] More files --- README.md | 109 +++++++++--------- about/index.md | 7 ++ getting-started/index.md | 10 ++ interactive-queries/index.md | 5 + other-components/index.md | 1 + querying/index.md | 6 + types/index.md | 3 + under-the-hood/contexture-client.md | 1 + under-the-hood/contexture-core.md | 6 +- .../contexture-elasticsearch.md | 1 + .../contexture-providers/contexture-mongo.md | 1 + .../index.md} | 0 under-the-hood/contexture-react.md | 1 + under-the-hood/design-principles.md | 20 ++++ under-the-hood/index.md | 50 +++----- 15 files changed, 128 insertions(+), 93 deletions(-) create mode 100644 about/index.md create mode 100644 getting-started/index.md create mode 100644 interactive-queries/index.md create mode 100644 other-components/index.md create mode 100644 querying/index.md create mode 100644 under-the-hood/contexture-client.md create mode 100644 under-the-hood/contexture-providers/contexture-elasticsearch.md create mode 100644 under-the-hood/contexture-providers/contexture-mongo.md rename under-the-hood/{contexture-providers.md => contexture-providers/index.md} (100%) create mode 100644 under-the-hood/contexture-react.md diff --git a/README.md b/README.md index 22ea070..7a6c87c 100644 --- a/README.md +++ b/README.md @@ -5,62 +5,63 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. ## Table of Contents -- [About Contexture]() - - [What is Contexture]() - - [Map of Repos]() - - [Brief History]() - - [Alternatives & Benchmarks]() -- [Getting Started]() - - [Project Setup & Use]() - - [Connecting to Elasticsearch]() - - [Connecting to Other Databases]() - - [Simple Search Box]() - - [Your First Filter]() - - [Discovering the Database]() - - [IMDB Index]() -- [Querying]() - - Contexture Core. - - Contexture DSL. - - Available providers. -- [Interactive Queries]() - - Contexture Client. - - Introduction to Reactors. +- [About Contexture](about/index.md) + - [What is Contexture](about/what-is-contexture.md) + - [Map of Repos](about/map-of-repos.md) + - [Brief History](about/brief-history.md) + - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) +- [Getting Started](getting-started/index.md) + - [Project Setup & Use](getting-started/setup.md) + - [Connecting to Elasticsearch](getting-started/connecting.md) + - [Connecting to Other Databases](getting-started/connecting-other-databases.md) + - [Simple Search Box](getting-started/simple-search-box.md) + - [Your First Filter](getting-started/your-first-filter.md) + - [Discovering the Database](getting-started/discovering-the-database.md) + - [IMDB Index](getting-started/IMDB-example.md) +- [Querying](querying/index.md) + - [Contexture Core](querying/contexture-core.md) + - [Contexture DSL](querying/contexture-dsl.md) + - [Available providers](querying/available-providers.md) +- [Interactive Queries](interactive-queries/index.md) + - [Contexture Client](interactive-queries/contexture-client.md) + - [Introduction to Reactors](interactive-queries/reactors.md) - [Types and Type Components](types/index.md) - [DIY Types](types/diy-types.md) - [How to Write a Type](types/diy-types.md#how-to-wite-a-type) - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) - [The Example Types](types/diy-types.md#the-example-types) - - [ElasticSearch Example Types]() - - [Mongo Example Types]() - - [Available React Components for our Types]() -- [Other Components]() -- [Managing State]() - - MobX - - Redux (Coming Soon) -- [Theming]() -- [Recommendations]() - - [Architecture]() - - Client vs Server. - - Type definitions. - - Scaling. - - [Server Side Searches]() - - [Searching On an Endpoint]() - - [Caching]() - - [Client Side Searches]() - - [Click to Search]() - - [Real Time Searches]() - - [Cascading]() -- [Under the Hood]() - - [Design Principles]() - - Contexture Core. - - Contexture initializer. - - Providers. - - Types. - - Reactors in detail. - - Contexture ElasticSearch - - Contexture Mongo - - Contexture Client - - Contexture React -- [Examples]() -- [Contributing Guide]() -- [License]() + - [ElasticSearch Example Types](types/elasticsearch-example-types.md) + - [Mongo Example Types](types/mongo-example-types.md) + - [Available React Components for Types](types/example-types-react-components.md) +- [Other Components](other-components/index.md) +- [Managing State](managing-state/index.md) + - [MobX](managing-state/mobx.md) + - [Redux (Coming Soon)](managing-state/redux.md) +- [Theming](theming/index.md) +- [Recommendations](recommendations/index.md) + - [Architecture](recommendations/architecture.md) + - [Client vs Server](recommendations/architecture.md#client-vs-server) + - [Type definitions](recommendations/architecture.md#type-definitions) + - [Scaling](recommendations/architecture.md#scaling) + - [Server Side Searches](recommendations/server-side-searches.md) + - [Searching On an Endpoint](recommendations/server-side-searches.md#searching-on-an-endpoint) + - [Caching](recommendations/server-side-searches.md#caching) + - [Client Side Searches](recommendations/client-side-searches.md) + - [Click to Search](recommendations/client-side-searches.md#click-to-search) + - [Real Time Searches](recommendations/client-side-searches.md#real-time-searches) + - [Cascading](recommendations/client-side-searches.md#cascading) +- [Under the Hood](under-the-hood/index.md) + - [Design Principles](under-the-hood/design-principles.md) + - [Contexture Core](under-the-hood/contexture-core.md) + - [Default Export](under-the-hood/contexture-core.md#default-export) + - [The Algorithm](under-the-hood/contexture-core.md#the-algorithm) + - [Utility Functions](under-the-hood/contexture-core.md#utility-functions) + - [Contexture Providers](under-the-hood/contexture-providers/index.md) + - [Contexture ElasticSearch](under-the-hood/contexture-providers/) + - [Contexture Mongo](under-the-hood/contexture-providers/) + - [Contexture Client](under-the-hood/contexture-client.md) + - [Reactors in detail](under-the-hood/contexture-client.md#reactors-in-detail) + - [Contexture React](under-the-hood/contexture-react.md) +- [Examples](examples/index.md) +- [Contributing Guide](contributing-guide/index.md) +- [License](LICENSE) diff --git a/about/index.md b/about/index.md new file mode 100644 index 0000000..b0c576f --- /dev/null +++ b/about/index.md @@ -0,0 +1,7 @@ +# About Contexture + +Table of Contents: +- [What is Contexture](what-is-contexture.md) +- [Map of Repos](map-of-repos.md) +- [Brief History](brief-history.md) +- [Alternatives & Benchmarks](alternatives-benchmarks.md) diff --git a/getting-started/index.md b/getting-started/index.md new file mode 100644 index 0000000..6ff2c28 --- /dev/null +++ b/getting-started/index.md @@ -0,0 +1,10 @@ +# Getting Started + +Table of Contents: +- [Project Setup & Use](getting-started/setup.md) +- [Connecting to Elasticsearch](getting-started/connecting.md) +- [Connecting to Other Databases](getting-started/connecting-other-databases.md) +- [Simple Search Box](getting-started/simple-search-box.md) +- [Your First Filter](getting-started/your-first-filter.md) +- [Discovering the Database](getting-started/discovering-the-database.md) +- [IMDB Index](getting-started/IMDB-example.md) diff --git a/interactive-queries/index.md b/interactive-queries/index.md new file mode 100644 index 0000000..0a1295d --- /dev/null +++ b/interactive-queries/index.md @@ -0,0 +1,5 @@ +# Interactive Queries + +Table of Contents: +- [Contexture Client](contexture-client.md) +- [Introduction to Reactors](reactors.md) diff --git a/other-components/index.md b/other-components/index.md new file mode 100644 index 0000000..548e3bf --- /dev/null +++ b/other-components/index.md @@ -0,0 +1 @@ +# Other Components diff --git a/querying/index.md b/querying/index.md new file mode 100644 index 0000000..6acefd5 --- /dev/null +++ b/querying/index.md @@ -0,0 +1,6 @@ +# Querying + +Table of Contents: +- [Contexture Core](contexture-core.md) +- [Contexture DSL](contexture-dsl.md) +- [Available providers](available-providers.md) diff --git a/types/index.md b/types/index.md index f8ff6eb..1fc022b 100644 --- a/types/index.md +++ b/types/index.md @@ -5,3 +5,6 @@ Table of Contents: - [How to Write a Type](diy-types.md#how-to-wite-a-type) - [How to Write a UI Component for a Type](diy-types.md#how-to-write-a-ui-component-for-a-type) - [The Example Types](diy-types.md#the-example-types) +- [ElasticSearch Example Types](elasticsearch-example-types.md) +- [Mongo Example Types](mongo-example-types.md) +- [Available React Components for Types](example-types-react-components.md) diff --git a/under-the-hood/contexture-client.md b/under-the-hood/contexture-client.md new file mode 100644 index 0000000..6652437 --- /dev/null +++ b/under-the-hood/contexture-client.md @@ -0,0 +1 @@ +# Contexture Client diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md index 8637b00..43fa311 100644 --- a/under-the-hood/contexture-core.md +++ b/under-the-hood/contexture-core.md @@ -4,12 +4,10 @@ The core of Contexture is a package of its own. Located at [github.com/smartproc With this in mind, let's get some specifications. -## contexture's default export +## Default Export Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. -### Arguments and Return Values - The first argument is expected to be a plain JavaScript Object with two keys: - `providers`: Should be an object where each key will have a [Contexture Provider](#TODO). @@ -56,3 +54,5 @@ _Coming soon..._ - State flags. - Add filters. - DFS initialization. + +## Utility Functions diff --git a/under-the-hood/contexture-providers/contexture-elasticsearch.md b/under-the-hood/contexture-providers/contexture-elasticsearch.md new file mode 100644 index 0000000..d7a3556 --- /dev/null +++ b/under-the-hood/contexture-providers/contexture-elasticsearch.md @@ -0,0 +1 @@ +# Contexture ElasticSearch diff --git a/under-the-hood/contexture-providers/contexture-mongo.md b/under-the-hood/contexture-providers/contexture-mongo.md new file mode 100644 index 0000000..ec8a9c0 --- /dev/null +++ b/under-the-hood/contexture-providers/contexture-mongo.md @@ -0,0 +1 @@ +# Contexture Mongo diff --git a/under-the-hood/contexture-providers.md b/under-the-hood/contexture-providers/index.md similarity index 100% rename from under-the-hood/contexture-providers.md rename to under-the-hood/contexture-providers/index.md diff --git a/under-the-hood/contexture-react.md b/under-the-hood/contexture-react.md new file mode 100644 index 0000000..32f6e8d --- /dev/null +++ b/under-the-hood/contexture-react.md @@ -0,0 +1 @@ +# Contexture React diff --git a/under-the-hood/design-principles.md b/under-the-hood/design-principles.md index b1c1b6a..714a7c4 100644 --- a/under-the-hood/design-principles.md +++ b/under-the-hood/design-principles.md @@ -1 +1,21 @@ # Design Principles + +- Intentionally stateless. +- Detached from state management tools. +- Can work well with state management tools. +- Modern ES6+ +- Non-strict functional programming. +- Configuration based architecture. +- Focus on a very extensible small core. +- Small DSL. + - Database agnostic. + - Isomorphic Tree State. + - Optimized for database discovery. + - Optimized for advanced search interfaces. + - Aiming to be effective for arbitrarily complex database indexes. + - Simplicity over performance. +- Reaction management. + - Tree walking. + - State flags. + - What updates. + - Pauses. diff --git a/under-the-hood/index.md b/under-the-hood/index.md index 1bd5328..2a2244e 100644 --- a/under-the-hood/index.md +++ b/under-the-hood/index.md @@ -1,36 +1,14 @@ -# Under The Hood - -## Design Principles - -- Intentionally stateless. -- Detached from state management tools. -- Can work well with state management tools. -- Modern ES6+ -- Non-strict functional programming. -- Configuration based architecture. -- Focus on a very extensible small core. -- Small DSL. - - Database agnostic. - - Isomorphic Tree State. - - Optimized for database discovery. - - Optimized for advanced search interfaces. - - Aiming to be effective for arbitrarily complex database indexes. - - Simplicity over performance. -- Reaction management. - - Tree walking. - - State flags. - - What updates. - - Pauses. - -## DSL - -## Contexture Core -- Contexture Default Export. - - Arguments and return values. - - The Algorithm. -- Core Utils. - -## Contexture Providers - - Design Structure - - Contexture ElasticSearch - - Contexture Mongo +# Under the Hood + +Table of Contents: +- [Design Principles](design-principles.md) +- [Contexture Core](contexture-core.md) + - [Default Export](contexture-core.md#default-export) + - [The Algorithm](contexture-core.md#the-algorithm) + - [Utility Functions](contexture-core.md#utility-functions) +- [Contexture Providers](index.md) + - [Contexture ElasticSearch]() + - [Contexture Mongo]() +- [Contexture Client](contexture-client.md) + - [Reactors in detail](contexture-client.md#reactors-in-detail) +- [Contexture React](contexture-react.md) From 5c91c0bb959a380f80b1fed7fdfbb7dac13548a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 01:32:25 +0000 Subject: [PATCH 013/150] More links --- under-the-hood/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/under-the-hood/index.md b/under-the-hood/index.md index 2a2244e..f3fe71b 100644 --- a/under-the-hood/index.md +++ b/under-the-hood/index.md @@ -6,9 +6,9 @@ Table of Contents: - [Default Export](contexture-core.md#default-export) - [The Algorithm](contexture-core.md#the-algorithm) - [Utility Functions](contexture-core.md#utility-functions) -- [Contexture Providers](index.md) - - [Contexture ElasticSearch]() - - [Contexture Mongo]() +- [Contexture Providers](contexture-providers/index.md) + - [Contexture ElasticSearch](contexture-providers/contexture-elasticsearch.md) + - [Contexture Mongo](contexture-providers/contexture-mongo.md) - [Contexture Client](contexture-client.md) - [Reactors in detail](contexture-client.md#reactors-in-detail) - [Contexture React](contexture-react.md) From 655ab2a71df278e74ec4c320e2f16d63c6f67b36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 01:34:47 +0000 Subject: [PATCH 014/150] Parent link? --- types/index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/types/index.md b/types/index.md index 1fc022b..d3644c3 100644 --- a/types/index.md +++ b/types/index.md @@ -1,4 +1,6 @@ -# Types and Type Components +_[Parent](../README.md)_ + +# Types and Type Components Table of Contents: - [DIY Types](diy-types.md) From 9eb2c901cea08e44fa59c7289c31bd5d5c9a8962 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 01:35:37 +0000 Subject: [PATCH 015/150] Link? --- types/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/types/index.md b/types/index.md index d3644c3..1d28bc4 100644 --- a/types/index.md +++ b/types/index.md @@ -1,4 +1,4 @@ -_[Parent](../README.md)_ +[🔗 Parent](../README.md) # Types and Type Components From 1c13fb491959807756366657c7a9b6177108a1e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 01:37:56 +0000 Subject: [PATCH 016/150] Leftwards arrow --- types/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/types/index.md b/types/index.md index 1d28bc4..f06aea5 100644 --- a/types/index.md +++ b/types/index.md @@ -1,4 +1,4 @@ -[🔗 Parent](../README.md) +[↩ Parent](../README.md) # Types and Type Components From a97fdd488b080282c81028afa2c1ce0f03c69e0f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 01:40:23 +0000 Subject: [PATCH 017/150] More parents --- about/index.md | 4 +++- getting-started/index.md | 4 +++- interactive-queries/index.md | 4 +++- other-components/index.md | 4 +++- querying/index.md | 4 +++- under-the-hood/contexture-providers/index.md | 4 +++- under-the-hood/index.md | 4 +++- 7 files changed, 21 insertions(+), 7 deletions(-) diff --git a/about/index.md b/about/index.md index b0c576f..197b950 100644 --- a/about/index.md +++ b/about/index.md @@ -1,4 +1,6 @@ -# About Contexture +[↩ Parent](../README.md) + +# About Contexture Table of Contents: - [What is Contexture](what-is-contexture.md) diff --git a/getting-started/index.md b/getting-started/index.md index 6ff2c28..fd4bed6 100644 --- a/getting-started/index.md +++ b/getting-started/index.md @@ -1,4 +1,6 @@ -# Getting Started +[↩ Parent](../README.md) + +# Getting Started Table of Contents: - [Project Setup & Use](getting-started/setup.md) diff --git a/interactive-queries/index.md b/interactive-queries/index.md index 0a1295d..b3e91f2 100644 --- a/interactive-queries/index.md +++ b/interactive-queries/index.md @@ -1,4 +1,6 @@ -# Interactive Queries +[↩ Parent](../README.md) + +# Interactive Queries Table of Contents: - [Contexture Client](contexture-client.md) diff --git a/other-components/index.md b/other-components/index.md index 548e3bf..c87304f 100644 --- a/other-components/index.md +++ b/other-components/index.md @@ -1 +1,3 @@ -# Other Components +[↩ Parent](../README.md) + + # Other Components diff --git a/querying/index.md b/querying/index.md index 6acefd5..dd86eeb 100644 --- a/querying/index.md +++ b/querying/index.md @@ -1,4 +1,6 @@ -# Querying +[↩ Parent](../README.md) + + # Querying Table of Contents: - [Contexture Core](contexture-core.md) diff --git a/under-the-hood/contexture-providers/index.md b/under-the-hood/contexture-providers/index.md index 25fb01b..66c7fd6 100644 --- a/under-the-hood/contexture-providers/index.md +++ b/under-the-hood/contexture-providers/index.md @@ -1,4 +1,6 @@ -# Contexture Providers +[↩ Parent](../index.md) + + # Contexture Providers The _Contexture Providers_ are an abstraction layer around the databases that can be targeted by any given search tree. Each Provider is a function that expects the following parameters: diff --git a/under-the-hood/index.md b/under-the-hood/index.md index f3fe71b..fb3bc52 100644 --- a/under-the-hood/index.md +++ b/under-the-hood/index.md @@ -1,4 +1,6 @@ -# Under the Hood +[↩ Parent](../README.md) + + # Under the Hood Table of Contents: - [Design Principles](design-principles.md) From 9064250c0986cdf1564d10cdb776ab6c40fcc975 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 02:30:58 +0000 Subject: [PATCH 018/150] Bit of better grammar --- README.md | 2 +- types/diy-types.md | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 7a6c87c..b5f2af5 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Querying](querying/index.md) - [Contexture Core](querying/contexture-core.md) - [Contexture DSL](querying/contexture-dsl.md) - - [Available providers](querying/available-providers.md) + - [Available Providers](querying/available-providers.md) - [Interactive Queries](interactive-queries/index.md) - [Contexture Client](interactive-queries/contexture-client.md) - [Introduction to Reactors](interactive-queries/reactors.md) diff --git a/types/diy-types.md b/types/diy-types.md index 51b9987..7f138ab 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -9,9 +9,9 @@ However, even if our types are focused on the different search interfaces we provide, our API is designed to allow you to build any type you might need for any other possible use case you might encounter. -We believe in making a generic framework so you can be creative on your -search solutions. Because of that, we will start this document by explaining -how to build your own types. +We believe that making a generic framework will allow users to be +creative on their search solutions. Because of that, we will start this +document by explaining how to build your own types. ## How to Wite a Type @@ -23,13 +23,13 @@ JavaScript Object value with one or more of the following properties: | Property | Type | Params (Type) | Return Value Type | What it does | | --- | --- | --- | --- | --- | -| `hasValue` | Function | NodeSome Examples Object) | Boolean | Allows Contexture to know wether or not to process this search. | +| `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | | `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | -| `result` | Function | Node (Object), Search (Function) | Promise | Allows you to run a direct search for this type before Comtexture sends the full seaech with the whole tree. | +| `result` | Function | Node (Object), Search (Function) | Promise | Allows running a direct search for this type before Comtexture sends the full seaech with the whole tree. | -Once you have your type written, you can use it by sending it to an -existing [Contexture Provider](#TODO). It should look more or less -like: +Once you have written a type, you can use it by sending it to an +existing [Contexture Provider](../querying/available-providers.md). It +should look more or less like: ```javascript let myType = { From 2386fec889dc9b37dc8a7fdf8982b52efd5c5c06 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 21:55:36 +0000 Subject: [PATCH 019/150] Context Tree --- README.md | 4 ++++ types/diy-types.md | 10 +++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index b5f2af5..4e718e0 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Available Providers](querying/available-providers.md) - [Interactive Queries](interactive-queries/index.md) - [Contexture Client](interactive-queries/contexture-client.md) + - [Contexture Tree](interactive-queries/contexture-client.md#contexture-tree) - [Introduction to Reactors](interactive-queries/reactors.md) - [Types and Type Components](types/index.md) - [DIY Types](types/diy-types.md) @@ -60,7 +61,10 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Contexture ElasticSearch](under-the-hood/contexture-providers/) - [Contexture Mongo](under-the-hood/contexture-providers/) - [Contexture Client](under-the-hood/contexture-client.md) + - [Lenses](under-the-hood/contexture-client.md#lenses) - [Reactors in detail](under-the-hood/contexture-client.md#reactors-in-detail) + - [Serialization](under-the-hood/contexture-client.md#serialization) + - [Traversals](under-the-hood/contexture-client.md#traversals) - [Contexture React](under-the-hood/contexture-react.md) - [Examples](examples/index.md) - [Contributing Guide](contributing-guide/index.md) diff --git a/types/diy-types.md b/types/diy-types.md index 7f138ab..fd3f45f 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -5,9 +5,9 @@ used to perform a wide variety of different searches. Each type we offer also has a respective component in our `contexture-react` repo. We've made these components so you can quickstart your search interfaces! -However, even if our types are focused on the different search interfaces -we provide, our API is designed to allow you to build any type you might need -for any other possible use case you might encounter. +Even if our types are focused on the different search interfaces we +provide, our API is designed to allow you to build any type you might +need for any other possible use case you might encounter. We believe that making a generic framework will allow users to be creative on their search solutions. Because of that, we will start this @@ -15,6 +15,10 @@ document by explaining how to build your own types. ## How to Wite a Type +Writing a new single type is about writing two plain JavaScript Objects: +- One which is sent to the [Contexture Provider](../querying/available-providers.md). +- Another one which is sent to the initialization of the [Contexture Tree](../interactive-queries/contexture-client.md#contexture-tree). + ### Provider Type Writing a type is as simple as exposing any valid string property name From 49e456e7d9b366da61412dcdee8b43f6097480d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 22:42:54 +0000 Subject: [PATCH 020/150] Seems better --- README.md | 4 +- types/diy-types.md | 84 ++++++++++++++----- ...ypes.md => elasticsearch-example-types.md} | 7 +- types/index.md | 2 +- types/mongo-example-types.md | 6 ++ types/react-components.md | 6 ++ 6 files changed, 85 insertions(+), 24 deletions(-) rename types/{contexture-elasticsearch-example-types.md => elasticsearch-example-types.md} (86%) create mode 100644 types/mongo-example-types.md create mode 100644 types/react-components.md diff --git a/README.md b/README.md index 4e718e0..43620ed 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Available Providers](querying/available-providers.md) - [Interactive Queries](interactive-queries/index.md) - [Contexture Client](interactive-queries/contexture-client.md) - - [Contexture Tree](interactive-queries/contexture-client.md#contexture-tree) + - [Context Tree](interactive-queries/contexture-client.md#context-tree) - [Introduction to Reactors](interactive-queries/reactors.md) - [Types and Type Components](types/index.md) - [DIY Types](types/diy-types.md) @@ -33,7 +33,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [The Example Types](types/diy-types.md#the-example-types) - [ElasticSearch Example Types](types/elasticsearch-example-types.md) - [Mongo Example Types](types/mongo-example-types.md) - - [Available React Components for Types](types/example-types-react-components.md) + - [Available React Components for Types](types/react-components.md) - [Other Components](other-components/index.md) - [Managing State](managing-state/index.md) - [MobX](managing-state/mobx.md) diff --git a/types/diy-types.md b/types/diy-types.md index fd3f45f..b3a705d 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -1,4 +1,7 @@ -# DIY Types +[↩ Parent: README.md](../README.md) +[↩ Previous: Types and Type Components](index.md) + +# DIY Types The Contexture ecosystem provides a defined list of types that can be used to perform a wide variety of different searches. Each type we @@ -17,13 +20,15 @@ document by explaining how to build your own types. Writing a new single type is about writing two plain JavaScript Objects: - One which is sent to the [Contexture Provider](../querying/available-providers.md). -- Another one which is sent to the initialization of the [Contexture Tree](../interactive-queries/contexture-client.md#contexture-tree). +- Another one which is sent to the initialization of the [Context Tree](../interactive-queries/contexture-client.md#context-tree). ### Provider Type -Writing a type is as simple as exposing any valid string property name -on the types object of a specific provider, then assigning a plain -JavaScript Object value with one or more of the following properties: +Initializing [Contexture Core](../querying/contexture-core.md) +requires you to send a bunch of types per provider. A type on any +provider is just a valid string property name on the object that is +sent, accompanied with a corresponding value of a plain JavaScript +Object with one or more of the following properties: | Property | Type | Params (Type) | Return Value Type | What it does | | --- | --- | --- | --- | --- | @@ -58,12 +63,12 @@ able to keep the same required node properties, thus the same `hasValue`. This allows us to provide the same API for several types, and re-use code even if we switch the target database of the search. -Once you have a provider type defined for one or more providers, you -should write the same type for `contexture-client`. +Once you have a type defined for one or more providers, you should +write the same type for `contexture-client`. ### Contexture Client Type -Contexture Client already provides a bunch of types based on our +Contexture Client already provides a some types based on our `Example Types` (more on that later.) These type definitions help the client understand how a specific node affects every other node or itself. @@ -78,14 +83,15 @@ might need for each one of the following properties: | `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | The example types are already included in any instantiation -of Contexture Client's Contexture Tree. However, you can extend them -with any type you need simply by complementing the exposed -`exampleTypes` with your own. Here's an example where we initialize -a `ContextureTree` with the available `exampleTypes`, and our new `myType`: +of Contexture Client's Contexture Tree. However, you can add any type +you need simply by extending the exposed `exampleTypes` with your own. +In the following snippet, we initialize a `ContextureTree` with the +available `exampleTypes`, and our new `myType`: ```javascript import * as ContextureClient from 'contexture-client' -let tree = ContextureTree({ + +let tree = ContextureClient.ContextTree({ types: { ...ContextureClient.exampleTypes, myType: { @@ -98,19 +104,52 @@ let tree = ContextureTree({ } } } +}, { + // Here we will have the underlying tree }) ``` ## How to Write a UI Component for a Type -Writing a ussr interface for any type can be as simple as writing an -HTML or JSX Element that will render or write into any property of the -type, for example, using our custom type `myType`, we could write an -input field that, onChange, will write the field's value into the -`requiredProperty`. For example: +Writing a user interface for any type can be as simple as writing an +HTML or JSX Element that will render or modify any property of the +any node of an existing Contexture Tree, for example, using our custom +type `myType`, we could write an input field that, onChange, will +write the field's value into the `requiredProperty`. For example: ```javascript -let Component = node => ( +// This is ES6+ and JSX + +import * as ContextureClient from 'contexture-client' + +let tree = ContextureClient.ContextTree( + { + service: myService, + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } + }, { + key: 'root', + join: 'and', + children: [{ + key: 'myNode', + type: 'myType', + }] + } +) + +let node = tree.getNode(['root', 'myNode']) + +let Component = ({ node }) => ( { @@ -120,7 +159,10 @@ let Component = node => ( ) ``` -- Managing State +Now that you have a component you can render it and play with it, but +the component won't render by itself. If you want to see examples of +custom components with automatic updates, please look at our [Managing +State Guide](../managing-state/index.md). ## The Example Types @@ -134,3 +176,5 @@ serve as a guide to build any other type you might need for your application. You can also **extend our example types**, simply by assigning new types as properties on the objects exposed by `contexture-elasticsearch` and `contexture-mongo`. + +[Next ⃕: ElasticSearch Example Types](elasticsearch-example-types.md) diff --git a/types/contexture-elasticsearch-example-types.md b/types/elasticsearch-example-types.md similarity index 86% rename from types/contexture-elasticsearch-example-types.md rename to types/elasticsearch-example-types.md index 2afe3be..fb45f11 100644 --- a/types/contexture-elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -1,4 +1,7 @@ -# ElasticSearch Example Types +[↩ Parent: README.md](../README.md) +[↩ Previous: DIY Types](diy-types.md) + +# ElasticSearch Example Types Contexture is designed to target any database you might need. However, so far we have only inplemented database providers for the only @@ -25,3 +28,5 @@ Here's the definition of this type: | Property Name | Type | Description | | --- | --- | --- | | field | String | Name of the field that we will include or exclude from the search. | + +[Next ⃕: Mongo Example Types](mongo-example-types.md) diff --git a/types/index.md b/types/index.md index f06aea5..0179ba3 100644 --- a/types/index.md +++ b/types/index.md @@ -9,4 +9,4 @@ Table of Contents: - [The Example Types](diy-types.md#the-example-types) - [ElasticSearch Example Types](elasticsearch-example-types.md) - [Mongo Example Types](mongo-example-types.md) -- [Available React Components for Types](example-types-react-components.md) +- [Available React Components for Types](react-components.md) diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md new file mode 100644 index 0000000..1b9c7a4 --- /dev/null +++ b/types/mongo-example-types.md @@ -0,0 +1,6 @@ +[↩ Parent: README.md](../README.md) +[↩ Previous: ElasticSearch Example Types](elasticsearch-example-types.md) + +# MongoDB Example Types + +[Next ⃕: Available React Components for Types](react-components.md) diff --git a/types/react-components.md b/types/react-components.md new file mode 100644 index 0000000..5586252 --- /dev/null +++ b/types/react-components.md @@ -0,0 +1,6 @@ +[↩ Parent: README.md](../README.md) +[↩ Previous: Mongo Example Types](mongo-example-types.md) + +# MongoDB Example Types + +[Next ⃕: Available React Components for Types](react-components.md) From 6671cc32ed8730f473e143284e15223b96bd56fb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 22:46:23 +0000 Subject: [PATCH 021/150] changes --- README.md | 26 ++++++++++----------- about/{index.md => README.md} | 0 getting-started/{index.md => README.md} | 0 interactive-queries/{index.md => README.md} | 0 other-components/{index.md => README.md} | 0 querying/{index.md => README.md} | 0 types/{index.md => README.md} | 2 +- types/diy-types.md | 2 +- types/elasticsearch-example-types.md | 2 +- types/mongo-example-types.md | 2 +- types/react-components.md | 2 +- under-the-hood/{index.md => README.md} | 0 12 files changed, 18 insertions(+), 18 deletions(-) rename about/{index.md => README.md} (100%) rename getting-started/{index.md => README.md} (100%) rename interactive-queries/{index.md => README.md} (100%) rename other-components/{index.md => README.md} (100%) rename querying/{index.md => README.md} (100%) rename types/{index.md => README.md} (90%) rename under-the-hood/{index.md => README.md} (100%) diff --git a/README.md b/README.md index 43620ed..63dc7f0 100644 --- a/README.md +++ b/README.md @@ -5,12 +5,12 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. ## Table of Contents -- [About Contexture](about/index.md) +- [About Contexture](about/README.md) - [What is Contexture](about/what-is-contexture.md) - [Map of Repos](about/map-of-repos.md) - [Brief History](about/brief-history.md) - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) -- [Getting Started](getting-started/index.md) +- [Getting Started](getting-started/README.md) - [Project Setup & Use](getting-started/setup.md) - [Connecting to Elasticsearch](getting-started/connecting.md) - [Connecting to Other Databases](getting-started/connecting-other-databases.md) @@ -18,15 +18,15 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Your First Filter](getting-started/your-first-filter.md) - [Discovering the Database](getting-started/discovering-the-database.md) - [IMDB Index](getting-started/IMDB-example.md) -- [Querying](querying/index.md) +- [Querying](querying/README.md) - [Contexture Core](querying/contexture-core.md) - [Contexture DSL](querying/contexture-dsl.md) - [Available Providers](querying/available-providers.md) -- [Interactive Queries](interactive-queries/index.md) +- [Interactive Queries](interactive-queries/README.md) - [Contexture Client](interactive-queries/contexture-client.md) - [Context Tree](interactive-queries/contexture-client.md#context-tree) - [Introduction to Reactors](interactive-queries/reactors.md) -- [Types and Type Components](types/index.md) +- [Types and Type Components](types/README.md) - [DIY Types](types/diy-types.md) - [How to Write a Type](types/diy-types.md#how-to-wite-a-type) - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) @@ -34,12 +34,12 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [ElasticSearch Example Types](types/elasticsearch-example-types.md) - [Mongo Example Types](types/mongo-example-types.md) - [Available React Components for Types](types/react-components.md) -- [Other Components](other-components/index.md) -- [Managing State](managing-state/index.md) +- [Other Components](other-components/README.md) +- [Managing State](managing-state/README.md) - [MobX](managing-state/mobx.md) - [Redux (Coming Soon)](managing-state/redux.md) -- [Theming](theming/index.md) -- [Recommendations](recommendations/index.md) +- [Theming](theming/README.md) +- [Recommendations](recommendations/README.md) - [Architecture](recommendations/architecture.md) - [Client vs Server](recommendations/architecture.md#client-vs-server) - [Type definitions](recommendations/architecture.md#type-definitions) @@ -51,13 +51,13 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Click to Search](recommendations/client-side-searches.md#click-to-search) - [Real Time Searches](recommendations/client-side-searches.md#real-time-searches) - [Cascading](recommendations/client-side-searches.md#cascading) -- [Under the Hood](under-the-hood/index.md) +- [Under the Hood](under-the-hood/README.md) - [Design Principles](under-the-hood/design-principles.md) - [Contexture Core](under-the-hood/contexture-core.md) - [Default Export](under-the-hood/contexture-core.md#default-export) - [The Algorithm](under-the-hood/contexture-core.md#the-algorithm) - [Utility Functions](under-the-hood/contexture-core.md#utility-functions) - - [Contexture Providers](under-the-hood/contexture-providers/index.md) + - [Contexture Providers](under-the-hood/contexture-providers/README.md) - [Contexture ElasticSearch](under-the-hood/contexture-providers/) - [Contexture Mongo](under-the-hood/contexture-providers/) - [Contexture Client](under-the-hood/contexture-client.md) @@ -66,6 +66,6 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Serialization](under-the-hood/contexture-client.md#serialization) - [Traversals](under-the-hood/contexture-client.md#traversals) - [Contexture React](under-the-hood/contexture-react.md) -- [Examples](examples/index.md) -- [Contributing Guide](contributing-guide/index.md) +- [Examples](examples/README.md) +- [Contributing Guide](contributing-guide/README.md) - [License](LICENSE) diff --git a/about/index.md b/about/README.md similarity index 100% rename from about/index.md rename to about/README.md diff --git a/getting-started/index.md b/getting-started/README.md similarity index 100% rename from getting-started/index.md rename to getting-started/README.md diff --git a/interactive-queries/index.md b/interactive-queries/README.md similarity index 100% rename from interactive-queries/index.md rename to interactive-queries/README.md diff --git a/other-components/index.md b/other-components/README.md similarity index 100% rename from other-components/index.md rename to other-components/README.md diff --git a/querying/index.md b/querying/README.md similarity index 100% rename from querying/index.md rename to querying/README.md diff --git a/types/index.md b/types/README.md similarity index 90% rename from types/index.md rename to types/README.md index 0179ba3..b4b97ee 100644 --- a/types/index.md +++ b/types/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # Types and Type Components diff --git a/types/diy-types.md b/types/diy-types.md index b3a705d..507af83 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -1,4 +1,4 @@ -[↩ Parent: README.md](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: Types and Type Components](index.md) # DIY Types diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index fb45f11..8c58407 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -1,4 +1,4 @@ -[↩ Parent: README.md](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: DIY Types](diy-types.md) # ElasticSearch Example Types diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md index 1b9c7a4..ac94b32 100644 --- a/types/mongo-example-types.md +++ b/types/mongo-example-types.md @@ -1,4 +1,4 @@ -[↩ Parent: README.md](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: ElasticSearch Example Types](elasticsearch-example-types.md) # MongoDB Example Types diff --git a/types/react-components.md b/types/react-components.md index 5586252..e34cac0 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -1,4 +1,4 @@ -[↩ Parent: README.md](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: Mongo Example Types](mongo-example-types.md) # MongoDB Example Types diff --git a/under-the-hood/index.md b/under-the-hood/README.md similarity index 100% rename from under-the-hood/index.md rename to under-the-hood/README.md From 42cb98962e045cc32960579c991d1ca0212c2ab2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 22:48:15 +0000 Subject: [PATCH 022/150] more changes --- types/diy-types.md | 2 +- under-the-hood/README.md | 2 +- under-the-hood/contexture-providers/{index.md => README.md} | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) rename under-the-hood/contexture-providers/{index.md => README.md} (97%) diff --git a/types/diy-types.md b/types/diy-types.md index 507af83..20902ce 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -1,5 +1,5 @@ [↩ Parent: Table of Contents](../README.md) -[↩ Previous: Types and Type Components](index.md) +[↩ Previous: Types and Type Components](README.md) # DIY Types diff --git a/under-the-hood/README.md b/under-the-hood/README.md index fb3bc52..a05224d 100644 --- a/under-the-hood/README.md +++ b/under-the-hood/README.md @@ -8,7 +8,7 @@ Table of Contents: - [Default Export](contexture-core.md#default-export) - [The Algorithm](contexture-core.md#the-algorithm) - [Utility Functions](contexture-core.md#utility-functions) -- [Contexture Providers](contexture-providers/index.md) +- [Contexture Providers](contexture-providers/README.md) - [Contexture ElasticSearch](contexture-providers/contexture-elasticsearch.md) - [Contexture Mongo](contexture-providers/contexture-mongo.md) - [Contexture Client](contexture-client.md) diff --git a/under-the-hood/contexture-providers/index.md b/under-the-hood/contexture-providers/README.md similarity index 97% rename from under-the-hood/contexture-providers/index.md rename to under-the-hood/contexture-providers/README.md index 66c7fd6..d297db9 100644 --- a/under-the-hood/contexture-providers/index.md +++ b/under-the-hood/contexture-providers/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../index.md) +[↩ Parent](../README.md) # Contexture Providers From c8ffa78eb08365bc07dd3e6e4729113ab00e6251 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 22:50:07 +0000 Subject: [PATCH 023/150] Better arrows --- types/diy-types.md | 2 +- types/elasticsearch-example-types.md | 2 +- types/mongo-example-types.md | 2 +- types/react-components.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/types/diy-types.md b/types/diy-types.md index 20902ce..267b7a2 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -177,4 +177,4 @@ application. You can also **extend our example types**, simply by assigning new types as properties on the objects exposed by `contexture-elasticsearch` and `contexture-mongo`. -[Next ⃕: ElasticSearch Example Types](elasticsearch-example-types.md) +[↪ Next: ElasticSearch Example Types](elasticsearch-example-types.md) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index 8c58407..469f1df 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -29,4 +29,4 @@ Here's the definition of this type: | --- | --- | --- | | field | String | Name of the field that we will include or exclude from the search. | -[Next ⃕: Mongo Example Types](mongo-example-types.md) +[↪ Next: Mongo Example Types](mongo-example-types.md) diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md index ac94b32..5ddd4cd 100644 --- a/types/mongo-example-types.md +++ b/types/mongo-example-types.md @@ -3,4 +3,4 @@ # MongoDB Example Types -[Next ⃕: Available React Components for Types](react-components.md) +[↪ Next: Available React Components for Types](react-components.md) diff --git a/types/react-components.md b/types/react-components.md index e34cac0..9af9a83 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -3,4 +3,4 @@ # MongoDB Example Types -[Next ⃕: Available React Components for Types](react-components.md) +[↪ Next: Available React Components for Types](react-components.md) From 980cc3ca612145ebf6505bde4448b41aa50e756b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 22:51:16 +0000 Subject: [PATCH 024/150] correct next for react-components.md --- types/react-components.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/types/react-components.md b/types/react-components.md index 9af9a83..2826873 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -3,4 +3,4 @@ # MongoDB Example Types -[↪ Next: Available React Components for Types](react-components.md) +[↪ Next: Other Components](../other-components/README.md) From 6a8847ecf9496a71b347c2aa25e6c34234b9ec41 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Sat, 26 May 2018 22:52:52 +0000 Subject: [PATCH 025/150] Better Parrents --- about/README.md | 2 +- getting-started/README.md | 2 +- interactive-queries/README.md | 2 +- other-components/README.md | 2 +- querying/README.md | 2 +- under-the-hood/README.md | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/about/README.md b/about/README.md index 197b950..ddb9292 100644 --- a/about/README.md +++ b/about/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # About Contexture diff --git a/getting-started/README.md b/getting-started/README.md index fd4bed6..f54f614 100644 --- a/getting-started/README.md +++ b/getting-started/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # Getting Started diff --git a/interactive-queries/README.md b/interactive-queries/README.md index b3e91f2..5f5b4b6 100644 --- a/interactive-queries/README.md +++ b/interactive-queries/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # Interactive Queries diff --git a/other-components/README.md b/other-components/README.md index c87304f..7cf1eb6 100644 --- a/other-components/README.md +++ b/other-components/README.md @@ -1,3 +1,3 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # Other Components diff --git a/querying/README.md b/querying/README.md index dd86eeb..092d6c4 100644 --- a/querying/README.md +++ b/querying/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # Querying diff --git a/under-the-hood/README.md b/under-the-hood/README.md index a05224d..bf0295a 100644 --- a/under-the-hood/README.md +++ b/under-the-hood/README.md @@ -1,4 +1,4 @@ -[↩ Parent](../README.md) +[↩ Parent: Table of Contents](../README.md) # Under the Hood From b093f46fdb0b94a3c1a38225bae4ca9b3ae06e64 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:12:13 +0000 Subject: [PATCH 026/150] First commit with some about content --- README.md | 1 + about/README.md | 1 + about/glossary-of-terms.md | 27 +++++++++++++++++++++ about/map-of-repos.md | 24 +++++++++++++++++++ about/what-is-contexture.md | 47 +++++++++++++++++++++++++++++++++++++ 5 files changed, 100 insertions(+) create mode 100644 about/glossary-of-terms.md create mode 100644 about/map-of-repos.md create mode 100644 about/what-is-contexture.md diff --git a/README.md b/README.md index 63dc7f0..28014e9 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [About Contexture](about/README.md) - [What is Contexture](about/what-is-contexture.md) + - [Glossary of Terms](about/glossary-of-terms.md) - [Map of Repos](about/map-of-repos.md) - [Brief History](about/brief-history.md) - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) diff --git a/about/README.md b/about/README.md index ddb9292..49cdb6c 100644 --- a/about/README.md +++ b/about/README.md @@ -4,6 +4,7 @@ Table of Contents: - [What is Contexture](what-is-contexture.md) +- [Glossary of Terms](glossary-of-terms.md) - [Map of Repos](map-of-repos.md) - [Brief History](brief-history.md) - [Alternatives & Benchmarks](alternatives-benchmarks.md) diff --git a/about/glossary-of-terms.md b/about/glossary-of-terms.md new file mode 100644 index 0000000..4321bee --- /dev/null +++ b/about/glossary-of-terms.md @@ -0,0 +1,27 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: What is Contexture](what-is-contexture.md) + +# Glossary of Terms + +## Contexture + +> NOUN +> 1. The fact or manner of being woven or linked together to form a +> connected whole. +> 1.1. A mass of things interwoven together; a fabric. +> 1.2. The putting together of words and sentences in connected +> composition; the construct of a text. +> 1.3. A connected literary structure; a continuous text. + +[Source](https://en.oxforddictionaries.com/definition/us/contexture). + +## Domain-Specific Language (DSL) + +> A domain-specific language (DSL) is a computer language specialized +> to a particular application domain. This is in contrast to a +> general-purpose language (GPL), which is broadly applicable across +> domains. + +[Source](https://en.wikipedia.org/wiki/Domain-specific_language). + +[↪ Next: Map of Repos](map-of-repos.md) diff --git a/about/map-of-repos.md b/about/map-of-repos.md new file mode 100644 index 0000000..f1d93df --- /dev/null +++ b/about/map-of-repos.md @@ -0,0 +1,24 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Glossary of Terms](glossary-of-terms.md) + +# Map of Repos + +The Contexture framework comes to life through a list of repositories +that individually specialize in some needed layer for our +architecture. Let's explore the repos we have so far: + +## Contexture core + +[github.com/smartprocure/contexture](https://github.com/smartprocure/contexture) +is where our main DSL processor lives. It is a very small layer that +ties everything together. This one receives the information about the +different search representations, about the databases involved, and +the DSL, then outputs the search results respective to each one of the +queries described in a copy of the received DSL. + +You can read more about the core here: +- [In the repository](https://github.com/smartprocure/contexture). +- In our [querying docs](../querying/contexture-core.md). +- In our [under the hood docs](../under-the-hood/contexture-core.md). + +[↪ Next: Brief History](brief-history.md) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md new file mode 100644 index 0000000..7fe5659 --- /dev/null +++ b/about/what-is-contexture.md @@ -0,0 +1,47 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: About Contexture](README.md) + +# Next + +People of the Internet, here we officialy introduce you to +`contexture`, our framework for building search interfaces. + +This framework is carefully designed to be a generic solution for a +universe of unlimited possible search interfaces. We've started with a +minimal set of repositories that are representative of tools that +empower our business, but that are intended to be merely examples. If +anything, our approaches are merely use cases, for the ultimate +potential of this tool is ultimately yours to take. + +A quick search over the Internet would reveal that the word +`contexture` means: `the fact or manner of being woven or linked +together to form a connected whole` or even `the putting together of +words and sentences in connected compoition; the construction of a +text`. + +With this word we are trying to expose not only our ultimate +intentions, but also more or less how the system is built. The way our +projects work is by a DSL that is used to gather different intended +search inputs, each one representing some useful abstraction of a +search filter (like a search input or a tags filter), then using the +values to process the DSL into one or more different database query +languages, running the underlying searches as optimized as possible, +then return these values on the respective sections of the DSL, so +that each result can update each one of the components of the user +interface. + +The canonical example of a Contexture Node is faceted search, where +you have a checkbox list that is both a filter (in the sense that it +restricts results based on the checked values) and an aggregation +(which shows the top n values that can be checked). Contexture allows +them to be nested in advanced searches with boolean joins like +`and`/`or`/`not`. + +This thought process will become more clear as we progress through the +docs. Hopefully some pages later it will become clear how we provide a +new perspective on building search interfaces, and perhaps even how +you can use it to power up your business just like we have been doing +for almost a decade. + +[↪ Next: Glossary of Terms](glossary-of-terms.md) +[↪ Or perhaps you want to jump to: Map of Repos](map-of-repos.md) From 5fd9c8c22fc7d7911c4a489085596358bf983c4d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:14:22 +0000 Subject: [PATCH 027/150] Small improvements --- about/glossary-of-terms.md | 2 +- about/map-of-repos.md | 2 +- about/what-is-contexture.md | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/about/glossary-of-terms.md b/about/glossary-of-terms.md index 4321bee..de2fe73 100644 --- a/about/glossary-of-terms.md +++ b/about/glossary-of-terms.md @@ -1,4 +1,4 @@ -[↩ Parent: Table of Contents](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: What is Contexture](what-is-contexture.md) # Glossary of Terms diff --git a/about/map-of-repos.md b/about/map-of-repos.md index f1d93df..a41f4d3 100644 --- a/about/map-of-repos.md +++ b/about/map-of-repos.md @@ -1,4 +1,4 @@ -[↩ Parent: Table of Contents](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: Glossary of Terms](glossary-of-terms.md) # Map of Repos diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 7fe5659..414d5fa 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -1,7 +1,7 @@ -[↩ Parent: Table of Contents](../README.md) +[↩ Parent: Table of Contents](../README.md) [↩ Previous: About Contexture](README.md) -# Next +# What Is Contexture People of the Internet, here we officialy introduce you to `contexture`, our framework for building search interfaces. @@ -43,5 +43,5 @@ new perspective on building search interfaces, and perhaps even how you can use it to power up your business just like we have been doing for almost a decade. -[↪ Next: Glossary of Terms](glossary-of-terms.md) +[↪ Next: Glossary of Terms](glossary-of-terms.md) [↪ Or perhaps you want to jump to: Map of Repos](map-of-repos.md) From e39106853d18b6911ea67e26bb18a03862604ad0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:16:21 +0000 Subject: [PATCH 028/150] Small improvement --- about/what-is-contexture.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 414d5fa..dcb10e6 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -38,8 +38,8 @@ them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. This thought process will become more clear as we progress through the -docs. Hopefully some pages later it will become clear how we provide a -new perspective on building search interfaces, and perhaps even how +docs. Hopefully, some pages later it will become clear how we provide +a new perspective on building search interfaces, and perhaps even how you can use it to power up your business just like we have been doing for almost a decade. From bf864b5e71c77c67478b19c31704155f1abba5c3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:29:15 +0000 Subject: [PATCH 029/150] Is this better? --- about/what-is-contexture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index dcb10e6..432fbed 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -9,7 +9,7 @@ People of the Internet, here we officialy introduce you to This framework is carefully designed to be a generic solution for a universe of unlimited possible search interfaces. We've started with a minimal set of repositories that are representative of tools that -empower our business, but that are intended to be merely examples. If +empower our business, but are intended to be merely examples. If anything, our approaches are merely use cases, for the ultimate potential of this tool is ultimately yours to take. From a3fa783b1d7f9ec5b5e72e26cad26089d02d54a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:30:37 +0000 Subject: [PATCH 030/150] This seems better --- about/what-is-contexture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 432fbed..0938099 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -26,7 +26,7 @@ search inputs, each one representing some useful abstraction of a search filter (like a search input or a tags filter), then using the values to process the DSL into one or more different database query languages, running the underlying searches as optimized as possible, -then return these values on the respective sections of the DSL, so +then returning these values on the respective sections of the DSL, so that each result can update each one of the components of the user interface. From 6be18c83861d07ca4ac07be5e912b250af1c48ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:32:45 +0000 Subject: [PATCH 031/150] Removed repetition of words starting with ultimate --- about/what-is-contexture.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 0938099..7277b5e 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -10,8 +10,8 @@ This framework is carefully designed to be a generic solution for a universe of unlimited possible search interfaces. We've started with a minimal set of repositories that are representative of tools that empower our business, but are intended to be merely examples. If -anything, our approaches are merely use cases, for the ultimate -potential of this tool is ultimately yours to take. +anything, our approaches are merely use cases, for the potential of +this tool is ultimately yours to take. A quick search over the Internet would reveal that the word `contexture` means: `the fact or manner of being woven or linked From b7d066f778a937f185597c79bd6124b087ad5f5a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 15:33:54 +0000 Subject: [PATCH 032/150] repeated merely to only --- about/what-is-contexture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 7277b5e..45d3beb 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -10,7 +10,7 @@ This framework is carefully designed to be a generic solution for a universe of unlimited possible search interfaces. We've started with a minimal set of repositories that are representative of tools that empower our business, but are intended to be merely examples. If -anything, our approaches are merely use cases, for the potential of +anything, our approaches are only use cases, for the potential of this tool is ultimately yours to take. A quick search over the Internet would reveal that the word From e827134080822ed85779ceac6c83cb0a328296de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 16:22:28 +0000 Subject: [PATCH 033/150] typo --- about/what-is-contexture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 45d3beb..2816ffc 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -16,7 +16,7 @@ this tool is ultimately yours to take. A quick search over the Internet would reveal that the word `contexture` means: `the fact or manner of being woven or linked together to form a connected whole` or even `the putting together of -words and sentences in connected compoition; the construction of a +words and sentences in connected composition; the construction of a text`. With this word we are trying to expose not only our ultimate From 9dc3f66ae9419ae9490310b2127b8fc9495c91ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 16:24:04 +0000 Subject: [PATCH 034/150] clear repeated at the end --- about/what-is-contexture.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 2816ffc..f4a10a2 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -38,10 +38,10 @@ them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. This thought process will become more clear as we progress through the -docs. Hopefully, some pages later it will become clear how we provide -a new perspective on building search interfaces, and perhaps even how -you can use it to power up your business just like we have been doing -for almost a decade. +docs. Hopefully, some pages later it will be easy to grasp how we +provide a new perspective on building search interfaces, and perhaps +even how you can use it to power up your business just like we have +been doing for almost a decade. [↪ Next: Glossary of Terms](glossary-of-terms.md) [↪ Or perhaps you want to jump to: Map of Repos](map-of-repos.md) From b59b3d9678da903d4357d8c8b4ae0013f931852f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 16:25:15 +0000 Subject: [PATCH 035/150] separated some quoted bulletpoints --- about/glossary-of-terms.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/about/glossary-of-terms.md b/about/glossary-of-terms.md index de2fe73..5c9cc07 100644 --- a/about/glossary-of-terms.md +++ b/about/glossary-of-terms.md @@ -7,10 +7,10 @@ > NOUN > 1. The fact or manner of being woven or linked together to form a -> connected whole. -> 1.1. A mass of things interwoven together; a fabric. +> connected whole. +> 1.1. A mass of things interwoven together; a fabric. > 1.2. The putting together of words and sentences in connected -> composition; the construct of a text. +> composition; the construct of a text. > 1.3. A connected literary structure; a continuous text. [Source](https://en.oxforddictionaries.com/definition/us/contexture). From b8b9572645d6be0f55f29df67680eed6bf8bf70e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 16:39:03 +0000 Subject: [PATCH 036/150] Info about contexture providers in our map of repos --- about/map-of-repos.md | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/about/map-of-repos.md b/about/map-of-repos.md index a41f4d3..b653340 100644 --- a/about/map-of-repos.md +++ b/about/map-of-repos.md @@ -7,7 +7,7 @@ The Contexture framework comes to life through a list of repositories that individually specialize in some needed layer for our architecture. Let's explore the repos we have so far: -## Contexture core +## Contexture Core [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture) is where our main DSL processor lives. It is a very small layer that @@ -18,7 +18,28 @@ queries described in a copy of the received DSL. You can read more about the core here: - [In the repository](https://github.com/smartprocure/contexture). -- In our [querying docs](../querying/contexture-core.md). -- In our [under the hood docs](../under-the-hood/contexture-core.md). +- In our [docs about querying (Contexture Core section)](../querying/contexture-core.md). +- In our [under the hood docs (Contexture Core section)](../under-the-hood/contexture-core.md). + +## Contexture Providers + +The Contexture Providers are the interfacing layer that ties the +Contexture DSL to the tartgetted databases. So far, we have only two +open source providers: + +- [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch), + and +- [contexture-mongo](https://github.com/smartprocure/contexture-mongo). + +If you are planning to use either ElasticSearch or MongoDB for your +project, the best way to get started is to use those repositories. +However, if you need to use Contexture with another database, you will +need to implement the provider yourself. We're looking for code +contributors, so please don't feel limited to the current available +tools. Help us grow together! + +You can read more about Contexture Providers here: +- In our [docs about querying (Available Providers section)](../querying/available-providers.md). +- In our [under the hood docs (Contexture Providers section)](../under-the-hood/contexture-providers/README.md). [↪ Next: Brief History](brief-history.md) From 691ccfda289b4749578567f92a4370c61105db9e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 17:35:12 +0000 Subject: [PATCH 037/150] Typo --- about/map-of-repos.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/map-of-repos.md b/about/map-of-repos.md index b653340..d1267be 100644 --- a/about/map-of-repos.md +++ b/about/map-of-repos.md @@ -24,7 +24,7 @@ You can read more about the core here: ## Contexture Providers The Contexture Providers are the interfacing layer that ties the -Contexture DSL to the tartgetted databases. So far, we have only two +Contexture DSL to the targeted databases. So far, we have only two open source providers: - [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch), From da89b894629ce6645a21a2b530c2dbf045a3b7be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 17:47:43 +0000 Subject: [PATCH 038/150] More on the map of repos --- about/map-of-repos.md | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/about/map-of-repos.md b/about/map-of-repos.md index d1267be..7f72b5f 100644 --- a/about/map-of-repos.md +++ b/about/map-of-repos.md @@ -5,7 +5,8 @@ The Contexture framework comes to life through a list of repositories that individually specialize in some needed layer for our -architecture. Let's explore the repos we have so far: +architecture. We will be using most (if not all) of these projects in +our upcoming pages. Let's explore the repos we have so far: ## Contexture Core @@ -19,7 +20,7 @@ queries described in a copy of the received DSL. You can read more about the core here: - [In the repository](https://github.com/smartprocure/contexture). - In our [docs about querying (Contexture Core section)](../querying/contexture-core.md). -- In our [under the hood docs (Contexture Core section)](../under-the-hood/contexture-core.md). +- Or with greater detail in our [under the hood docs (Contexture Core section)](../under-the-hood/contexture-core.md). ## Contexture Providers @@ -40,6 +41,29 @@ tools. Help us grow together! You can read more about Contexture Providers here: - In our [docs about querying (Available Providers section)](../querying/available-providers.md). -- In our [under the hood docs (Contexture Providers section)](../under-the-hood/contexture-providers/README.md). +- In with greater detail in our [under the hood docs (Contexture Providers section)](../under-the-hood/contexture-providers/README.md). + +## Contexture Client + +The Contexture Client is responsible for triggering behaviors on the +search interfaces by knowing what causes changes in one or more +elements of the search tree. It is the key piece of technology that +allows our search interfaces to work in real time. + +You can read more about the Contexture Client here: +- [In the repository](https://github.com/smartprocure/contexture-client). +- [In the repository] - In our [docs about querying (Contexture Client section)](../interactive-queries/contexture-client.md). +- Or with greater detail in our [under the hood docs (Contexture Client section)](../under-the-hood/contexture-client.md). + +## Contexture React + +The Contexture React repository holds a list of components that +facilitate building search interfaces. They are mainly graphical +representations of the types that exist on our Contexture Providers. + +You can read more about the Contexture React: +- [In the repository](https://github.com/smartprocure/contexture-client). +- In our guide for the [Available React Components for Types](./types/react-components.md). +- Or in greater detail in our [under the hood docs (Contexture React section)](under-the-hood/contexture-react.md). [↪ Next: Brief History](brief-history.md) From d1b023b1bb6dd54a1b440a69c17c1327448aea9f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 17:51:13 +0000 Subject: [PATCH 039/150] Not sure what happened here --- about/map-of-repos.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/map-of-repos.md b/about/map-of-repos.md index 7f72b5f..a3e7523 100644 --- a/about/map-of-repos.md +++ b/about/map-of-repos.md @@ -52,7 +52,7 @@ allows our search interfaces to work in real time. You can read more about the Contexture Client here: - [In the repository](https://github.com/smartprocure/contexture-client). -- [In the repository] - In our [docs about querying (Contexture Client section)](../interactive-queries/contexture-client.md). +- In our [docs about querying (Contexture Client section)](../interactive-queries/contexture-client.md). - Or with greater detail in our [under the hood docs (Contexture Client section)](../under-the-hood/contexture-client.md). ## Contexture React From 20eff318985611ca780e28dec4886ccd8f4b9411 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 17:55:28 +0000 Subject: [PATCH 040/150] Blank files about history and benchmarks --- about/alternatives-benchmarks.md | 8 ++++++++ about/brief-history.md | 8 ++++++++ 2 files changed, 16 insertions(+) create mode 100644 about/alternatives-benchmarks.md create mode 100644 about/brief-history.md diff --git a/about/alternatives-benchmarks.md b/about/alternatives-benchmarks.md new file mode 100644 index 0000000..f36db33 --- /dev/null +++ b/about/alternatives-benchmarks.md @@ -0,0 +1,8 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Brief History](brief-history.md) + +# Brief History + + TODO + +[↪ Next: Getting Started](../getting-started/README.md) diff --git a/about/brief-history.md b/about/brief-history.md new file mode 100644 index 0000000..fdd3b2f --- /dev/null +++ b/about/brief-history.md @@ -0,0 +1,8 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Map of Repos](map-of-repos.md) + +# Brief History + + TODO + +[↪ Next: Alternatives & Benchmarks](alternatives-benchmarks.md) From a526b2c29ad681cac69b670179305ee4386b43d3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 17:56:17 +0000 Subject: [PATCH 041/150] Corrected a title --- about/alternatives-benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/alternatives-benchmarks.md b/about/alternatives-benchmarks.md index f36db33..cb4f186 100644 --- a/about/alternatives-benchmarks.md +++ b/about/alternatives-benchmarks.md @@ -1,7 +1,7 @@ [↩ Parent: Table of Contents](../README.md) [↩ Previous: Brief History](brief-history.md) -# Brief History +# Alternatives & Benchmarks TODO From 929476dc003e91a6a625746ed957674a9a975272 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 18:43:15 +0000 Subject: [PATCH 042/150] More on the setup --- README.md | 9 ++++- getting-started/README.md | 21 ++++++++---- getting-started/setup.md | 72 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+), 8 deletions(-) create mode 100644 getting-started/setup.md diff --git a/README.md b/README.md index 28014e9..52cdec2 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,14 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) - [Getting Started](getting-started/README.md) - [Project Setup & Use](getting-started/setup.md) - - [Connecting to Elasticsearch](getting-started/connecting.md) + - [Install Node 9 and NPM](getting-started/setup.md#installing-node-9-and-npm) + - [Install Contexture](getting-started/setup.md#installing-contexture) + - [Install Contexture Client](getting-started/setup.md#installing-contexture-client) + - [Install Contexture ElasticSearch](getting-started/setup.md#installing-contexture-elasticsearch) + - [Install Contexture Mongo](getting-started/setup.md#installing-contexture-mongo) + - [Install Contexture React](getting-started/setup.md#installing-contexture-react) + - [Write your first Contexture script](getting-started/setup.md#your-first-contexture-script) + - [Connecting to Elasticsearch & MongoDB](getting-started/connecting.md) - [Connecting to Other Databases](getting-started/connecting-other-databases.md) - [Simple Search Box](getting-started/simple-search-box.md) - [Your First Filter](getting-started/your-first-filter.md) diff --git a/getting-started/README.md b/getting-started/README.md index f54f614..a08654f 100644 --- a/getting-started/README.md +++ b/getting-started/README.md @@ -3,10 +3,17 @@ # Getting Started Table of Contents: -- [Project Setup & Use](getting-started/setup.md) -- [Connecting to Elasticsearch](getting-started/connecting.md) -- [Connecting to Other Databases](getting-started/connecting-other-databases.md) -- [Simple Search Box](getting-started/simple-search-box.md) -- [Your First Filter](getting-started/your-first-filter.md) -- [Discovering the Database](getting-started/discovering-the-database.md) -- [IMDB Index](getting-started/IMDB-example.md) +- [Project Setup & Use](setup.md) + - [Install Node 9 and NPM](setup.md#installing-node-9-and-npm) + - [Install Contexture](setup.md#installing-contexture) + - [Install Contexture Client](setup.md#installing-contexture-client) + - [Install Contexture ElasticSearch](setup.md#installing-contexture-elasticsearch) + - [Install Contexture Mongo](setup.md#installing-contexture-mongo) + - [Install Contexture React](setup.md#installing-contexture-react) + - [Write your first Contexture script](setup.md#your-first-contexture-script) +- [Connecting to Elasticsearch & MongoDB](connecting.md) +- [Connecting to Other Databases](connecting-other-databases.md) +- [Simple Search Box](simple-search-box.md) +- [Your First Filter](your-first-filter.md) +- [Discovering the Database](discovering-the-database.md) +- [IMDB Index](IMDB-example.md) diff --git a/getting-started/setup.md b/getting-started/setup.md new file mode 100644 index 0000000..edadf75 --- /dev/null +++ b/getting-started/setup.md @@ -0,0 +1,72 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Getting Started](README.md) + +# Project Setup & Use + +The whole Contexture Framework is available through NPM. It doesn't +have any extra dependency besides Node 9+. In this quick guide, we'll +achieve the following goals: + +- [Install Node 9 and NPM](./setup.md#installing-node-9-and-npm) +- [Install Contexture](./setup.md#installing-contexture) +- [Install Contexture Client](./setup.md#installing-contexture-client) +- [Install Contexture ElasticSearch](./setup.md#installing-contexture-elasticsearch) +- [Install Contexture Mongo](./setup.md#installing-contexture-mongo) +- [Install Contexture React](./setup.md#installing-contexture-react) +- [Write your first Contexture script](./setup.md#your-first-contexture-script) + +## Installing Node 9 and NPM + +NodeJS 9 and NPM can be installed through [the list of previous +releases of NodeJS](https://nodejs.org/en/download/releases/). You +might get it working with Node 10 (if so, let us know). We haven't +fully upgraded to Node 10 yet, so until then, we encourage you to at +least have a working version of Node 9 in hand. + +An easy way to move from one version to another is with +[nvm](https://github.com/creationix/nvm). Here's a command you can run +to install nvm: + + curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash + +Please follow [nvm's README](https://github.com/creationix/nvm/blob/master/README.md) for more information. + +## Installing Contexture + +Once you have NodeJS and NPM installed, you'll need either a new +folder for your new project with Contexture, or to go to an existing +project, then run: + + npm install contexture + +## Installing Contexture Client + +To install the `contexture-client` you can also run the followign +command in your project's root folder: + + npm install contexture-client + +## Installing Contexture ElasticSearch + +To install the `contexture-elasticsearch` you can also run the followign +command in your project's root folder: + + npm install contexture-elasticsearch + +## Installing Contexture Mongo + +To install the `contexture-mongo` you can also run the followign +command in your project's root folder: + + npm install contexture-mongo + +## Installing Contexture React + +To install the `contexture-react` you can also run the followign +command in your project's root folder: + + npm install contexture-react + +## Your First Contexture Script + +[↪ Next: Connecting to ElasticSearch & MongoDB](connecting.md) From aba81a45dcea76e2ae3e620a702ab93acd2ea0f1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 20:52:50 +0000 Subject: [PATCH 043/150] Mostly the first script --- README.md | 5 +- getting-started/README.md | 4 +- getting-started/first-script.md | 188 ++++++++++++++++++++++++++++ getting-started/setup.md | 24 ++-- querying/README.md | 1 + under-the-hood/design-principles.md | 4 + 6 files changed, 211 insertions(+), 15 deletions(-) create mode 100644 getting-started/first-script.md diff --git a/README.md b/README.md index 52cdec2..cec5349 100644 --- a/README.md +++ b/README.md @@ -12,14 +12,14 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Brief History](about/brief-history.md) - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) - [Getting Started](getting-started/README.md) - - [Project Setup & Use](getting-started/setup.md) + - [Project Setup](getting-started/setup.md) - [Install Node 9 and NPM](getting-started/setup.md#installing-node-9-and-npm) - [Install Contexture](getting-started/setup.md#installing-contexture) - [Install Contexture Client](getting-started/setup.md#installing-contexture-client) - [Install Contexture ElasticSearch](getting-started/setup.md#installing-contexture-elasticsearch) - [Install Contexture Mongo](getting-started/setup.md#installing-contexture-mongo) - [Install Contexture React](getting-started/setup.md#installing-contexture-react) - - [Write your first Contexture script](getting-started/setup.md#your-first-contexture-script) + - [A First Contexture Script](getting-started/first-script.md) - [Connecting to Elasticsearch & MongoDB](getting-started/connecting.md) - [Connecting to Other Databases](getting-started/connecting-other-databases.md) - [Simple Search Box](getting-started/simple-search-box.md) @@ -30,6 +30,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Contexture Core](querying/contexture-core.md) - [Contexture DSL](querying/contexture-dsl.md) - [Available Providers](querying/available-providers.md) + - [Schemas](querying/schemas.md) - [Interactive Queries](interactive-queries/README.md) - [Contexture Client](interactive-queries/contexture-client.md) - [Context Tree](interactive-queries/contexture-client.md#context-tree) diff --git a/getting-started/README.md b/getting-started/README.md index a08654f..5558cd6 100644 --- a/getting-started/README.md +++ b/getting-started/README.md @@ -3,14 +3,14 @@ # Getting Started Table of Contents: -- [Project Setup & Use](setup.md) +- [Project Setup](setup.md) - [Install Node 9 and NPM](setup.md#installing-node-9-and-npm) - [Install Contexture](setup.md#installing-contexture) - [Install Contexture Client](setup.md#installing-contexture-client) - [Install Contexture ElasticSearch](setup.md#installing-contexture-elasticsearch) - [Install Contexture Mongo](setup.md#installing-contexture-mongo) - [Install Contexture React](setup.md#installing-contexture-react) - - [Write your first Contexture script](setup.md#your-first-contexture-script) +- [A First Contexture Script](first-script.md) - [Connecting to Elasticsearch & MongoDB](connecting.md) - [Connecting to Other Databases](connecting-other-databases.md) - [Simple Search Box](simple-search-box.md) diff --git a/getting-started/first-script.md b/getting-started/first-script.md new file mode 100644 index 0000000..2ca8d27 --- /dev/null +++ b/getting-started/first-script.md @@ -0,0 +1,188 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Project Setup](setup.md) + +## Your First Contexture Script + +With everything installed, let's see a simple script that runs a +simple one-time search: + +```javascript +let contexture = require('contexture') + +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} + +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} + +let result = await contexture({ + schemas, + providers: { + mongo: require('contexture-mongo')({ + types: require('contexture-mongo/types')(), + getClient: () => ({ + // Fake mongo client. + // For this example we only care about + // collection().aggregate([...]).toArray() being a promise. + collection: name => ({ + aggregate: aggregations => ({ + toArray: async () => ['Unrealistic result example'] + }) + }) + }) + }) + }, +}, searchTree) + +console.log(result.children[1].context.response.results) + +// Explore it yourself! +result +``` + +You can also try this same code in Runkit here: + + +## What it does + +```javascript +let contexture = require('contexture') +``` +We start by requiring `contexture`. + +```javascript +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} +``` +Immediatly afterwards, we define our schemas. For this specific +example, we are going to emulate doing a search on a single collection +of a Mongo database. We define `collectionName` as the name of this +arbitrary collection. The `schemas` object ends up containing a single +schema with a key being `collectionNameSchema`, which is going to be +supported by a single provider `mongo`, from which we'll be looking +for the `collectionName` collection. We'll learn more about the +schemas later on, in [Querying → Schemas](../querying/schemas.md). + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} +``` +Our next step is to define a simple search query using Contexture DSL. +This search query will have a mandatory root group, which among other +features indicates which schema we will be running this query on +(`collectionNameSchema`). This node will have two children. The first +children is going to be our search query. This query will be a `text` +type query. We're indicating that we want to run a plain text search +that will try to match any record which `name` contains the word +`value`. The next node has the `results` type. This is where the +results will be written once the search runs. + +You can read more about these topics in the following links: + +- [Querying](../querying/README.md). +- [Types and Type Components](../types/README.md). +- [Mongo Example Types](../types/mongo-example-types.md). + +```javascript +let result = await contexture({ + schemas, + providers: { + mongo: require('contexture-mongo')({ + types: require('contexture-mongo/types')(), + getClient: () => ({ + // Fake mongo client. + // For this example we only care about + // collection().aggregate([...]).toArray() being a promise. + collection: name => ({ + aggregate: aggregations => ({ + toArray: async () => ['Unrealistic result example'] + }) + }) + }) + }) + }, +}, searchTree) +``` + +The next thing we do is to actually run a one-time search. In this +case, we `await` for `contexture`, passing along some very important +parameters. First, we pass the schemas we previously defined. Then, we +pass a providers object, which has the provider for `mongo`. This +`mongo` key will be matched against the `mongo` key that we defined in +the schemas, so you could change this property name to something +entirely different. When we send the mongo provider, we assign the +result of the initialization of `contexture-mongo`, where we send the +`contxture-mongo/types` and a `getClient` function. In a real +schenario, you would just send the object that results of +`require('mongodb')`. However, to provide an exceutable example in the +browser, we've made a very small Mock of MongoDB where we will return +a same object for any collection call, which will only allow fake +aggregations, which will have a promise `toArray` function that will +return our `Unrealistic result example`. + +Keep in mind that since `contexture` returns a promise, you can change +`await contexture({ /* .. */ })` to `contexture({ /* .. */ }).then()`, +but you'll need to move the code that we have after the await call +into the function that is passed on the `.then()` call. + +You can read more about these concepts here: +- [Querying](../querying/README.md). +- [Contexture Providers](../under-the-hood/contexture-providers/README.md). +- Contexture Core's [Default Export](../under-the-hood/contexture-core.md#default-export). + +```javascript +console.log(result.children[1].context.response.results) + +// Explore it yourself! +result +``` + +Finally, we can use our search result! The output of the search will +be added to a copy of the original search query. The location of this +result will be in the children of the `root` node that has the +`results` type, within a `context` object, within a `response` object, +on a `results` property. This whole object is going to be exposed in +the _runkit_ example for you to play with it. For more information, +you can dive in here: + +- [Contexture DSL](../querying/contexture-dsl.md). +- [Design Principles](../under-the-hood/design-principles.md). +- [Contexture Core](../under-the-hood/contexture-core.md) + +[↪ Next: Connecting to ElasticSearch & MongoDB](connecting.md) diff --git a/getting-started/setup.md b/getting-started/setup.md index edadf75..aff2f54 100644 --- a/getting-started/setup.md +++ b/getting-started/setup.md @@ -1,10 +1,10 @@ [↩ Parent: Table of Contents](../README.md) [↩ Previous: Getting Started](README.md) -# Project Setup & Use +# Project Setup The whole Contexture Framework is available through NPM. It doesn't -have any extra dependency besides Node 9+. In this quick guide, we'll +have any extra dependency besides Node 9. In this quick guide, we'll achieve the following goals: - [Install Node 9 and NPM](./setup.md#installing-node-9-and-npm) @@ -13,7 +13,11 @@ achieve the following goals: - [Install Contexture ElasticSearch](./setup.md#installing-contexture-elasticsearch) - [Install Contexture Mongo](./setup.md#installing-contexture-mongo) - [Install Contexture React](./setup.md#installing-contexture-react) -- [Write your first Contexture script](./setup.md#your-first-contexture-script) + +**Note:** as you progress through our documentation, you'll discover +that in some cases, you will need only one or two of these +repositories. This page just provides the fastest start we can come up +with. ## Installing Node 9 and NPM @@ -37,36 +41,34 @@ Once you have NodeJS and NPM installed, you'll need either a new folder for your new project with Contexture, or to go to an existing project, then run: - npm install contexture + npm install --save contexture ## Installing Contexture Client To install the `contexture-client` you can also run the followign command in your project's root folder: - npm install contexture-client + npm install --save contexture-client ## Installing Contexture ElasticSearch To install the `contexture-elasticsearch` you can also run the followign command in your project's root folder: - npm install contexture-elasticsearch + npm install --save contexture-elasticsearch ## Installing Contexture Mongo To install the `contexture-mongo` you can also run the followign command in your project's root folder: - npm install contexture-mongo + npm install --save contexture-mongo ## Installing Contexture React To install the `contexture-react` you can also run the followign command in your project's root folder: - npm install contexture-react - -## Your First Contexture Script + npm install --save contexture-react -[↪ Next: Connecting to ElasticSearch & MongoDB](connecting.md) +[↪ Next: A First Contexture Script](first-script.md) diff --git a/querying/README.md b/querying/README.md index 092d6c4..726a838 100644 --- a/querying/README.md +++ b/querying/README.md @@ -6,3 +6,4 @@ Table of Contents: - [Contexture Core](contexture-core.md) - [Contexture DSL](contexture-dsl.md) - [Available providers](available-providers.md) +- [Schemas](schemas.md) diff --git a/under-the-hood/design-principles.md b/under-the-hood/design-principles.md index 714a7c4..8fb0b8c 100644 --- a/under-the-hood/design-principles.md +++ b/under-the-hood/design-principles.md @@ -19,3 +19,7 @@ - State flags. - What updates. - Pauses. +- Why types? +- Why a results type? +- Why schemas? +- Why providers? From 17cd0d65218e03ff3a6eee662162fc636b3a8e16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 29 May 2018 20:54:34 +0000 Subject: [PATCH 044/150] Important clarification --- getting-started/first-script.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/getting-started/first-script.md b/getting-started/first-script.md index 2ca8d27..ef27331 100644 --- a/getting-started/first-script.md +++ b/getting-started/first-script.md @@ -148,13 +148,14 @@ pass a providers object, which has the provider for `mongo`. This the schemas, so you could change this property name to something entirely different. When we send the mongo provider, we assign the result of the initialization of `contexture-mongo`, where we send the -`contxture-mongo/types` and a `getClient` function. In a real -schenario, you would just send the object that results of -`require('mongodb')`. However, to provide an exceutable example in the -browser, we've made a very small Mock of MongoDB where we will return -a same object for any collection call, which will only allow fake -aggregations, which will have a promise `toArray` function that will -return our `Unrealistic result example`. +`contxture-mongo/types` (which **needs to be called as a function once +required**) and a `getClient` function. In a real schenario, you would +just send the object that results of `require('mongodb')`. However, to +provide an exceutable example in the browser, we've made a very small +Mock of MongoDB where we will return a same object for any collection +call, which will only allow fake aggregations, which will have a +promise `toArray` function that will return our `Unrealistic result +example`. Keep in mind that since `contexture` returns a promise, you can change `await contexture({ /* .. */ })` to `contexture({ /* .. */ }).then()`, From 478d463fbe7d270ad3563e62c54bed80522e83fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 21:25:32 +0000 Subject: [PATCH 045/150] Connecting --- README.md | 2 + getting-started/README.md | 2 + getting-started/connecting.md | 310 ++++++++++++++++++++++++++++++++++ 3 files changed, 314 insertions(+) create mode 100644 getting-started/connecting.md diff --git a/README.md b/README.md index cec5349..4a9a873 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,8 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Install Contexture React](getting-started/setup.md#installing-contexture-react) - [A First Contexture Script](getting-started/first-script.md) - [Connecting to Elasticsearch & MongoDB](getting-started/connecting.md) + - [Connecting to ElasticSearch](getting-started/connecting.md#connecting-to-elasticsearch.md) + - [Connecting to MongoDB](getting-started/connecting.md#connecting-to-mongodb.md) - [Connecting to Other Databases](getting-started/connecting-other-databases.md) - [Simple Search Box](getting-started/simple-search-box.md) - [Your First Filter](getting-started/your-first-filter.md) diff --git a/getting-started/README.md b/getting-started/README.md index 5558cd6..d326132 100644 --- a/getting-started/README.md +++ b/getting-started/README.md @@ -12,6 +12,8 @@ Table of Contents: - [Install Contexture React](setup.md#installing-contexture-react) - [A First Contexture Script](first-script.md) - [Connecting to Elasticsearch & MongoDB](connecting.md) + - [Connecting to ElasticSearch](connecting.md#connecting-to-elasticsearch.md) + - [Connecting to MongoDB](connecting.md#connecting-to-mongodb.md) - [Connecting to Other Databases](connecting-other-databases.md) - [Simple Search Box](simple-search-box.md) - [Your First Filter](your-first-filter.md) diff --git a/getting-started/connecting.md b/getting-started/connecting.md new file mode 100644 index 0000000..fb56a9c --- /dev/null +++ b/getting-started/connecting.md @@ -0,0 +1,310 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Project Setup](setup.md) + +## Connecting to Elasticsearch & MongoDB + +Our primary use case for advanced search interfaces is to query data +that we store on ElasticSearch and MongoDB. Because of that, we +provide two contexture libraries for those databases. In this page +we'll examine how to use these repositories to connect to existing +Mongo databases and ElasticSearch indexes. + +### Connecting to ElasticSearch + +The followng code example shows how to connect to ElasticSearch: + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-elasticsearch') +let types = require('contexture-elasticsearch/types') +let elasticsearch = require('elasticsearch') +let AgentKeepAlive = require('agentkeepalive') + +let elasticClient = null +let getClient = async () => { + if (elasticClient) return elasticClient + elasticClient = elasticsearch.Client({ + minSockets: 1, + maxSockets: 20, + keepAlive: true, + createNodeAgent: (connection, config) => + new AgentKeepAlive(connection.makeAgentConfig(config)) + }) + return elasticClient +} + +let schemas = { + yourCustomSchemaName: { + elasticsearch: { + index: 'SomeIndex', + type: 'SomeType' + } + } +} + +let runSearch = Contexture({ + schemas, + providers: { + elasticsearch: provider({ + getClient, + request: { + headers: { + 'custom-header-app-name': 'my-app-sent-this' + } + }, + types: types() + }) + } +}) +``` + +The code above will provide a working search function `runSearch` that +will transform any given query into a working ElasticSearch query, +which will be sent to the database to retrieve the data. Let's examine +this code in greater detail. + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-elasticsearch') +let types = require('contexture-elasticsearch/types') +let elasticsearch = require('elasticsearch') +let AgentKeepAlive = require('agentkeepalive') +``` + +The first six lines are about requiring dependencies, the first +dependency being just `contexture`. The second one is our database +provider, `contexture-elasticsearch`. Then, we require the types; even +though you will probably have to write your own types, we have some +pre-defined types available at `contexture-elasticsearch/types`. +Lastly, we require `elasticsearch` and `agentkeepalive`, so we can +actually connect to ElasticSearch and keep it connected even if the +address becomes unreachable for a while. + +Please feel free to dig around these topics by following these links: + +- [Contexture Providers](under-the-hood/contexture-providers/README.md). +- [Types and Type Components](types/README.md). +- [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). +- [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). + + +```javascript +let elasticClient = null +let getClient = () => { + if (elasticClient) return elasticClient + elasticClient = elasticsearch.Client({ + minSockets: 1, + maxSockets: 20, + keepAlive: true, + createNodeAgent: (connection, config) => + new AgentKeepAlive(connection.makeAgentConfig(config)) + }) + return elasticClient +} +``` + +Now, we define a utility function to connect to ElasticSearch just +once. The idea here is to be able to re-use the same connection +instead of using new clients every time a search is executed. We do +this by declaring a function that will return `elasticClient` if it has +a truthy value, or otherwise instantiate a new elasticsearch client +with the given configuration. The configuration we are sending is +merely an example and shouldn't be used without understanding what is +being expected by the elasticsearch library. More on the following +links: + +- [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). + +```javascript +let schemas = { + elasticsearch: { + index: 'SomeIndex', + type: 'SomeType' + } +} +``` + +As shown above, the next thing we do is to define the schemas. This is +an object which properties are the names of each one of the schemas +that will be available for the contexture DSL. The schemas for the +ElasticSearch provider can specify any or all of the following +properties: + +| Option | Type | Description | Required | +| ------ | ---- | ----------- | -------- | +| `index` | `string` | Which ES index to use when querying | x | +| `type` | `string` | Which ES type to use when querying | | +| `summaryView` | `function` | Used by `results` to return a summary view instead of the whole document, (e.g. for indexes with many fields). Defaults to returning the `hit` property. | | +| `highlight` | `object` | Used by `results` to determine what fields to highlight, and whether or not they are `inline` (copied over inline on to the source) or `additional` (in a list of additional fields that matched) | | +| `forceExclude` | `array` | Used by `results` to extend the exclude fields provided on the search tree. The extension happens only if the results node has a `forceExclude` flag set to true. + +You can read more about these here: + +- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). +- [Schemas section on our Querying docs](../querying/schemas.md). + +```javascript +let runSearch = Contexture({ + schemas, + providers: { + elasticsearch: provider({ + getClient, + request: { + headers: { + 'custom-header-app-name': 'my-app-sent-this' + } + }, + types: types() + }) + } +}) +``` + +Once we have the schemas set, we can create our `runSearch` function. +For this purpose, we will be calling `Contexture` with just one +object. This object will have the `schemas`, and the `providers`. The +providers will host just one key/value, the one specific for +`elasticsearch`. The provider, which we required at the beginning of +the script, needs to be called with the `getClient` function we just +created and the `types()` that we got from the +`contexture-elasticsearch/types` repository. We also show that you can +customize the request headers by providing an object that includes the +headers keys and values. This `runSearch` function is ready to receive +search trees and write the results back! + +You can read more about these topics in the following links: + +- [Contexture Core's Default Export](../under-the-hood/contexture-core.md#default-export). +- [Contexture Providers in detail](under-the-hood/contexture-providers/README.md). + +This example and many other important details about +`contexture-elasticsearch` are accessible in the following links: + +- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). + +### Connecting to MongoDB + +The followng code example shows how to connect to MongoDB: + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient + +let schemas = { + yourCustomSchemaName: { + mongo: { + collection: 'SomeCollection' + } + } +} + +let runSearch = null + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + runSearch = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +The code above will provide a working search function `runSearch` that +will transform any given query into a working MongoDB query, +which will be sent to the database to retrieve the data. Let's examine +this code in greater detail. + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient +``` + +The first lines are about requiring dependencies, the first +dependency being just `contexture`. The second one is our database +provider, `contexture-mongo`. Then, we require the types; even +though you will probably have to write your own types, we have some +pre-defined types available at `contexture-mongo/types`. +Lastly, we require `mongodb` to get the much needed `MongoClient`, so +we can actually connect to the database. + +Please feel free to dig around these topics by following these links: + +- [Contexture Providers](under-the-hood/contexture-providers/README.md). +- [Types and Type Components](types/README.md). +- [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). + + +```javascript +let schemas = { + yourCustomSchemaName: { + mongo: { + collection: 'SomeCollection' + } + } +} +``` + +As shown above, the next thing we do is to define the schemas. This is +an object which properties are the names of each one of the schemas +that will be available for the contexture DSL. The schemas for the +MongoDB provider can specify any or all of the following +properties: + +| Option | Type | Description | Required | +| ------ | ---- | ----------- | -------- | +| `collection` | `string` | The MongoDB collection that will be used to run the queries | x | + +You can read more about these here: + +- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). +- [Schemas section on our Querying docs](../querying/schemas.md). + +```javascript +let runSearch = null + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + runSearch = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +Next we will need to connect to MongoDB. In the previous code, we +provide an example in which we connect to `mongodb://localhost:27017`, +and using a callback to the `connect` method, we are able to obtain +the database client that we need. Once we have this client, we can +actually create our `runSearch` function. For this purpose, we will +be calling `Contexture` with just one object. This object will have +the `schemas`, and the `providers`. The providers will host just one +key/value, the one specific for `mongo`. The provider, which +we required at the beginning of the script, needs to be called with +a `getClient` function that will just return the database client, and +the `types()` that we got from the `contexture-mongo/types` repository. +With these steps completed, we end up with a function that is ready to +receive search trees and write the results back! + +You can read more about these topics in the following links: + +- [Contexture Core's Default Export](../under-the-hood/contexture-core.md#default-export). +- [Contexture Providers in detail](under-the-hood/contexture-providers/README.md). + +This example and many other important details about +`contexture-mongo` are accessible in the following links: + +- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). + +[↪ Next: Connecting to Other Databases](connecting-other-databases.md) From f3df6ed8b8b7530a5e635100b09d79cb55151363 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 21:28:41 +0000 Subject: [PATCH 046/150] Is this better? --- getting-started/connecting.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/getting-started/connecting.md b/getting-started/connecting.md index fb56a9c..0989182 100644 --- a/getting-started/connecting.md +++ b/getting-started/connecting.md @@ -63,6 +63,8 @@ will transform any given query into a working ElasticSearch query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. +1. The Dependencies + ```javascript let Contexture = require('contexture') let provider = require('contexture-elasticsearch') @@ -220,6 +222,8 @@ will transform any given query into a working MongoDB query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. +1. The Dependencies + ```javascript let Contexture = require('contexture') let provider = require('contexture-mongo') From bdcac5e4aae43eaa915251872aed118107132834 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 21:31:10 +0000 Subject: [PATCH 047/150] Maybe this is better --- getting-started/connecting.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/getting-started/connecting.md b/getting-started/connecting.md index 0989182..fc92273 100644 --- a/getting-started/connecting.md +++ b/getting-started/connecting.md @@ -89,6 +89,7 @@ Please feel free to dig around these topics by following these links: - [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). - [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). +2. The ElasticSearch Client ```javascript let elasticClient = null @@ -117,6 +118,8 @@ links: - [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). +3. The Schemas + ```javascript let schemas = { elasticsearch: { @@ -145,6 +148,8 @@ You can read more about these here: - [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). - [Schemas section on our Querying docs](../querying/schemas.md). +4. Our Search Function + ```javascript let runSearch = Contexture({ schemas, @@ -245,6 +250,7 @@ Please feel free to dig around these topics by following these links: - [Types and Type Components](types/README.md). - [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). +2. The Schemas ```javascript let schemas = { @@ -271,6 +277,8 @@ You can read more about these here: - [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). - [Schemas section on our Querying docs](../querying/schemas.md). +3. The MongoDB Client & Our Search Function + ```javascript let runSearch = null From 2c61785a7acac88f30b5a9f78fbe434ba85bbd70 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 21:56:02 +0000 Subject: [PATCH 048/150] Concept for connecting to other databases --- getting-started/connecting-other-databases.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 getting-started/connecting-other-databases.md diff --git a/getting-started/connecting-other-databases.md b/getting-started/connecting-other-databases.md new file mode 100644 index 0000000..b51d080 --- /dev/null +++ b/getting-started/connecting-other-databases.md @@ -0,0 +1,19 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Connecting to Elasticsearch & MongoDB](connecting.md) + +## Connecting to Other Databases + +Contexture depends on it's providers to be able to know how to +translate from the Contexture DSL to the specific DSL that each +database needs. Because of this, to other databases you will need to +create a new Provider. + +Writing a new provider consists of writing a function (, or class or +anything other approach you might like,) that returns the following +properties: + +| Property Name | Type | Example Value | Description | +| --- | --- | --- | --- | +| `types` | Object | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo) | This will contain the [Contexture Types](../types/diy-types.md) for your custom Provider | + +[↪ Next: Simple Search Box](simple-search-box.md) From 12a1a5b6194822e44a7be73dfa05aff9b29d5518 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 23:07:05 +0000 Subject: [PATCH 049/150] Building your own provider & connecting to other databases --- README.md | 1 + getting-started/connecting-other-databases.md | 16 ++-- getting-started/connecting.md | 22 ++--- under-the-hood/README.md | 1 + under-the-hood/contexture-core.md | 8 +- under-the-hood/contexture-providers/README.md | 29 ------ .../building-your-own-provider.md | 92 +++++++++++++++++++ 7 files changed, 117 insertions(+), 52 deletions(-) delete mode 100644 under-the-hood/contexture-providers/README.md create mode 100644 under-the-hood/contexture-providers/building-your-own-provider.md diff --git a/README.md b/README.md index 4a9a873..6967685 100644 --- a/README.md +++ b/README.md @@ -71,6 +71,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Contexture Providers](under-the-hood/contexture-providers/README.md) - [Contexture ElasticSearch](under-the-hood/contexture-providers/) - [Contexture Mongo](under-the-hood/contexture-providers/) + - [Building Your Own Provider](under-the-hood/contexture-providers/building-your-own-provider.md) - [Contexture Client](under-the-hood/contexture-client.md) - [Lenses](under-the-hood/contexture-client.md#lenses) - [Reactors in detail](under-the-hood/contexture-client.md#reactors-in-detail) diff --git a/getting-started/connecting-other-databases.md b/getting-started/connecting-other-databases.md index b51d080..d58f802 100644 --- a/getting-started/connecting-other-databases.md +++ b/getting-started/connecting-other-databases.md @@ -5,15 +5,13 @@ Contexture depends on it's providers to be able to know how to translate from the Contexture DSL to the specific DSL that each -database needs. Because of this, to other databases you will need to -create a new Provider. +database needs. Because of this, to connect to other databases you +will need to create a new Provider. -Writing a new provider consists of writing a function (, or class or -anything other approach you might like,) that returns the following -properties: - -| Property Name | Type | Example Value | Description | -| --- | --- | --- | --- | -| `types` | Object | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo) | This will contain the [Contexture Types](../types/diy-types.md) for your custom Provider | +If you want to build your own provider, please follow this detailed +guide: [Building Your Own +Provider](under-the-hood/contexture-providers/building-your-own-provider.md). +Beware that it might get into the core of contexture, which is not +needed if you can take advantage of the existing providers. [↪ Next: Simple Search Box](simple-search-box.md) diff --git a/getting-started/connecting.md b/getting-started/connecting.md index fc92273..d4e2e6e 100644 --- a/getting-started/connecting.md +++ b/getting-started/connecting.md @@ -42,7 +42,7 @@ let schemas = { } } -let runSearch = Contexture({ +let search = Contexture({ schemas, providers: { elasticsearch: provider({ @@ -58,7 +58,7 @@ let runSearch = Contexture({ }) ``` -The code above will provide a working search function `runSearch` that +The code above will provide a working search function `search` that will transform any given query into a working ElasticSearch query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. @@ -151,7 +151,7 @@ You can read more about these here: 4. Our Search Function ```javascript -let runSearch = Contexture({ +let search = Contexture({ schemas, providers: { elasticsearch: provider({ @@ -167,7 +167,7 @@ let runSearch = Contexture({ }) ``` -Once we have the schemas set, we can create our `runSearch` function. +Once we have the schemas set, we can create our `search` function. For this purpose, we will be calling `Contexture` with just one object. This object will have the `schemas`, and the `providers`. The providers will host just one key/value, the one specific for @@ -176,7 +176,7 @@ the script, needs to be called with the `getClient` function we just created and the `types()` that we got from the `contexture-elasticsearch/types` repository. We also show that you can customize the request headers by providing an object that includes the -headers keys and values. This `runSearch` function is ready to receive +headers keys and values. This `search` function is ready to receive search trees and write the results back! You can read more about these topics in the following links: @@ -207,10 +207,10 @@ let schemas = { } } -let runSearch = null +let search = null MongoClient.connect('mongodb://localhost:27017', function(err, client) { - runSearch = Contexture({ + search = Contexture({ schemas, providers: { mongo: provider({ @@ -222,7 +222,7 @@ MongoClient.connect('mongodb://localhost:27017', function(err, client) { }) ``` -The code above will provide a working search function `runSearch` that +The code above will provide a working search function `search` that will transform any given query into a working MongoDB query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. @@ -280,10 +280,10 @@ You can read more about these here: 3. The MongoDB Client & Our Search Function ```javascript -let runSearch = null +let search = null MongoClient.connect('mongodb://localhost:27017', function(err, client) { - runSearch = Contexture({ + search = Contexture({ schemas, providers: { mongo: provider({ @@ -299,7 +299,7 @@ Next we will need to connect to MongoDB. In the previous code, we provide an example in which we connect to `mongodb://localhost:27017`, and using a callback to the `connect` method, we are able to obtain the database client that we need. Once we have this client, we can -actually create our `runSearch` function. For this purpose, we will +actually create our `search` function. For this purpose, we will be calling `Contexture` with just one object. This object will have the `schemas`, and the `providers`. The providers will host just one key/value, the one specific for `mongo`. The provider, which diff --git a/under-the-hood/README.md b/under-the-hood/README.md index bf0295a..ddcc8dc 100644 --- a/under-the-hood/README.md +++ b/under-the-hood/README.md @@ -11,6 +11,7 @@ Table of Contents: - [Contexture Providers](contexture-providers/README.md) - [Contexture ElasticSearch](contexture-providers/contexture-elasticsearch.md) - [Contexture Mongo](contexture-providers/contexture-mongo.md) + - [Building Your Own Provider](contexture-providers/building-your-own-provider.md) - [Contexture Client](contexture-client.md) - [Reactors in detail](contexture-client.md#reactors-in-detail) - [Contexture React](contexture-react.md) diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md index 43fa311..4703eb9 100644 --- a/under-the-hood/contexture-core.md +++ b/under-the-hood/contexture-core.md @@ -35,13 +35,15 @@ const search = Contexture({ ``` The other two parameters are the search tree (the [Contexture -DSL](#TODO)), and an optional object with the following properties: +DSL](#TODO)), and an optional object that will be sent along to the +provider's `runSearch` function as the first parameter, that can +contain any property, but that should at least contain the following +properties: | Option | Description | | ------ | ----------- | | `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info. | -| `onResult` | A callback which is called whenever a node finishes -producing it's results, which can be used to send partial results over websockets for example. | +| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example. | This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the [Contexture Types](#TODO). diff --git a/under-the-hood/contexture-providers/README.md b/under-the-hood/contexture-providers/README.md deleted file mode 100644 index d297db9..0000000 --- a/under-the-hood/contexture-providers/README.md +++ /dev/null @@ -1,29 +0,0 @@ -[↩ Parent](../README.md) - - # Contexture Providers - -The _Contexture Providers_ are an abstraction layer around the databases that can be targeted by any given search tree. Each Provider is a function that expects the following parameters: - -| Input Object Property | Meaning | -| ------- | ------- | -| `types` | [Contexture Types](#TODO) that apply for this specific provider | -| `getClient` | Function that will return the direct database API manager, such as the `mongodb` NPM package. | - -This initialization retutns an object with: - -| Return Object Property | Meaning | -| ------- | ------- | -| `types` | [Contexture Types](#TODO) that apply for this specific provider | -| `groupCombinator` | Function that will determine how to construct boolean operations on the search tree nodes, such as `and`, `or` and `not`. | - -With that in mind, let's explore our currently available providers: - -## contexture-mongo -- Where the package lives -- What it does -- list of types (not much detail, link to the details docs) - -## contexture-elasticsearch -- Where the package lives -- What it does -- list of types (not much detail, link to the details docs) diff --git a/under-the-hood/contexture-providers/building-your-own-provider.md b/under-the-hood/contexture-providers/building-your-own-provider.md new file mode 100644 index 0000000..c8ba146 --- /dev/null +++ b/under-the-hood/contexture-providers/building-your-own-provider.md @@ -0,0 +1,92 @@ +[↩ Parent: Under the Hood](../README.md) +[↩ Previous: Contexture Mongo](contexture-mongo.md) + +# Building Your Own Provider + +Writing a new provider consists of writing a function (or class or +any other approach you might like), that will receive a configuration +object with at least the following properties: + +| Property Name | Type | Description | +| --- | --- | --- | +| `types` | `Object` | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo). More information on our detailed docs about [Contexture Types](../types/diy-types.md) | +| `getClient` | `Function` | A function that returns the main database client that you will be using. | + +You might also add any new property if your provider needs it for +processing the searches. + +The main provider function is expected to return an Object with: + +| Property Name | Type | Description | +| --- | --- | --- | +| `types` | `Object` | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo). More information on our detailed docs about [Contexture Types](../types/diy-types.md) | +| `groupCombinator(group, filters)` | `Function` | A function that returns a valid query for grouping individual aggregations. More on that below. | +| `runSearch(options = {}, context, schema, filters, aggs)` | `Function -> Promise` | A function that will be responsible for running each one of the individual searches that the types in the Contexture DSL might indicate. More on that below. | + +### Group Combinator Function + +The `groupCombinator` function is used to transform an incoming query +into the underlying database's group combinator. It receives a `group` +Object that contains a property `join`, which indicates which boolean +operation wants to be used. The `join` values we use are `and`, `or` +and `not`, but you can specify any `join` value for your provider, as +long as you make sure the resulting query is valid. The result should +be a native database query structure with the given `filters` within a +boolean operator. For example, in ElasticSearch, this means: + +| Boolean Operation | ElasticSearch Combinator | +| --- | --- | +| `and` | `{ bool: { must: filters } }` | +| `or` | `{ bool: { should: filters, minimum_should_match: 1 } }` | +| `not` | `{ bool: { must_not: filters } }` | + +However, in MongoDB, this is how it works: + +| Boolean Operation | MongoDB Combinator | +| --- | --- | +| `and` | `{ $and: filters }` | +| `or` | `{ $or: filters }` | +| `not` | `{ $nor: filters }` | + +### Run Search Function + +The `runSearch` function will run each search that is indicated by the +Contexture DSL. It receives several parameters: + +| Property Name | Type | Description | +| --- | --- | --- | +| `options` | `Object` | Object that is sent as the last parameter to the main `Contexture` call. | +| `context` | `Object` | The current node that is triggering the search. | +| `schema` | `Object` | The schema that is related to this node or root. Usually assigned at the root of the search tree. | +| `filters` | `Array` | The filters that have resulted from parsing this node and it's children. | +| `aggs` | `Array` | Any other aggregation that might have been received from previous processing operations of the Contexture DSL. | + +In this function, you should call to the `getClient` function that the +provider originally received, then you should accomodate the incoming +`filters` and `aggs` into the final query that will be sent to the +client's aggregation method. Since you'll have the schema, you'll be +able to get the information you need for your specific database with +`schema.myDatabase.myProperty`. + +The final search query should be an Object and should be pushed into +the `context._meta.requests` array, for debugging purposes. Once you have +the result, you should also add it to the final query object, so that +it can be accessible by reading the context path, then +`_meta.requests[index].response`, where `index` is the position where +the final query lives. For example: + +```javascript +let request = { + /* My Database DSL Object */ +} + +context._meta.requests.push(request) + +let result = Promise.resolve(databaseClient(request)) + +return result.tap(results => { + request.response = results +}) +``` + +[↪ Next: Contexture Client](../contexture-client.md) From fe405d1b15b0985eacd8be4e7e03743d677050f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 23:08:07 +0000 Subject: [PATCH 050/150] Woopsie --- getting-started/connecting-other-databases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/getting-started/connecting-other-databases.md b/getting-started/connecting-other-databases.md index d58f802..091a243 100644 --- a/getting-started/connecting-other-databases.md +++ b/getting-started/connecting-other-databases.md @@ -10,7 +10,7 @@ will need to create a new Provider. If you want to build your own provider, please follow this detailed guide: [Building Your Own -Provider](under-the-hood/contexture-providers/building-your-own-provider.md). +Provider](../under-the-hood/contexture-providers/building-your-own-provider.md). Beware that it might get into the core of contexture, which is not needed if you can take advantage of the existing providers. From 9e9f69782afbd91c0e3b9e94436444567287b2df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 30 May 2018 23:39:56 +0000 Subject: [PATCH 051/150] Diagram --- about/what-is-contexture.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index f4a10a2..246771e 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -30,6 +30,8 @@ then returning these values on the respective sections of the DSL, so that each result can update each one of the components of the user interface. +![Diagram](https://i.imgur.com/L96DVYh.png) + The canonical example of a Contexture Node is faceted search, where you have a checkbox list that is both a filter (in the sense that it restricts results based on the checked values) and an aggregation From ccd58731cffa71268b1707585f6db48217e85ddb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 02:15:25 +0000 Subject: [PATCH 052/150] Refining the what-is-contexture.md file --- about/what-is-contexture.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 246771e..19b92a5 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -15,20 +15,22 @@ this tool is ultimately yours to take. A quick search over the Internet would reveal that the word `contexture` means: `the fact or manner of being woven or linked -together to form a connected whole` or even `the putting together of +together to form a connected whole` and also `the putting together of words and sentences in connected composition; the construction of a text`. -With this word we are trying to expose not only our ultimate -intentions, but also more or less how the system is built. The way our -projects work is by a DSL that is used to gather different intended -search inputs, each one representing some useful abstraction of a -search filter (like a search input or a tags filter), then using the -values to process the DSL into one or more different database query -languages, running the underlying searches as optimized as possible, -then returning these values on the respective sections of the DSL, so -that each result can update each one of the components of the user -interface. +Picking `contexture` as the name for this project means that we are +trying to expose not only our ultimate intentions, but also more or +less how the system is built. The way our projects work is by a DSL +that is used to gather different intended search inputs, each one +representing some useful abstraction of a search filter (like an input +where you can write a word to be searched, or another where you can +filter the search results by one or more options), then using the +values to process a DSL that will end up retrieving values from one or +more different databases, then returning these values on the +respective sections of the DSL, so that each result can update each +one of the components of the user interface. A more detailed +description is visible in the following diagram. ![Diagram](https://i.imgur.com/L96DVYh.png) @@ -42,7 +44,7 @@ them to be nested in advanced searches with boolean joins like This thought process will become more clear as we progress through the docs. Hopefully, some pages later it will be easy to grasp how we provide a new perspective on building search interfaces, and perhaps -even how you can use it to power up your business just like we have +even how you can use it to power up your business, just like we have been doing for almost a decade. [↪ Next: Glossary of Terms](glossary-of-terms.md) From cede6616cb83c6e10e9281d8e5fba90557ea4335 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 13:52:48 +0000 Subject: [PATCH 053/150] Advanced search representation --- about/what-is-contexture.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 19b92a5..e629c6f 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -41,6 +41,8 @@ restricts results based on the checked values) and an aggregation them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. +![Advanced Search Representation](https://i.imgur.com/8ui0t4h.png) + This thought process will become more clear as we progress through the docs. Hopefully, some pages later it will be easy to grasp how we provide a new perspective on building search interfaces, and perhaps From c8f9995666002f8a047a86a82b32286da7e3a51c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 13:54:17 +0000 Subject: [PATCH 054/150] Centering images --- about/what-is-contexture.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index e629c6f..6e3f89c 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -32,7 +32,7 @@ respective sections of the DSL, so that each result can update each one of the components of the user interface. A more detailed description is visible in the following diagram. -![Diagram](https://i.imgur.com/L96DVYh.png) +

![Diagram](https://i.imgur.com/L96DVYh.png)

The canonical example of a Contexture Node is faceted search, where you have a checkbox list that is both a filter (in the sense that it @@ -41,7 +41,7 @@ restricts results based on the checked values) and an aggregation them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. -![Advanced Search Representation](https://i.imgur.com/8ui0t4h.png) +

![Advanced Search Representation](https://i.imgur.com/8ui0t4h.png)

This thought process will become more clear as we progress through the docs. Hopefully, some pages later it will be easy to grasp how we From bb50b38a3dd4a9407ccb8ee59abae217bebd3291 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 13:55:24 +0000 Subject: [PATCH 055/150] Seems like we can't mix markdown with html --- about/what-is-contexture.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 6e3f89c..8d4ed61 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -32,7 +32,7 @@ respective sections of the DSL, so that each result can update each one of the components of the user interface. A more detailed description is visible in the following diagram. -

![Diagram](https://i.imgur.com/L96DVYh.png)

+

The canonical example of a Contexture Node is faceted search, where you have a checkbox list that is both a filter (in the sense that it @@ -41,7 +41,7 @@ restricts results based on the checked values) and an aggregation them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. -

![Advanced Search Representation](https://i.imgur.com/8ui0t4h.png)

+

This thought process will become more clear as we progress through the docs. Hopefully, some pages later it will be easy to grasp how we From 8420f7063c450d6a6ee212ca1a091ef99d0acd2a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 13:57:27 +0000 Subject: [PATCH 056/150] Slightly better picture --- about/what-is-contexture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md index 8d4ed61..71cbd9f 100644 --- a/about/what-is-contexture.md +++ b/about/what-is-contexture.md @@ -41,7 +41,7 @@ restricts results based on the checked values) and an aggregation them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. -

+

This thought process will become more clear as we progress through the docs. Hopefully, some pages later it will be easy to grasp how we From 405b5b22b9e841cab11219f39f26357a09f1a8ba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:26:14 +0000 Subject: [PATCH 057/150] Some more terms --- about/glossary-of-terms.md | 43 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/about/glossary-of-terms.md b/about/glossary-of-terms.md index 5c9cc07..1574bb1 100644 --- a/about/glossary-of-terms.md +++ b/about/glossary-of-terms.md @@ -24,4 +24,47 @@ [Source](https://en.wikipedia.org/wiki/Domain-specific_language). +## Search Query + +According to Wikipedia, `Search Query`, in the context of the `web`, refers to: + +> ...a query that a user enters into a web search engine to satisfy his or her +> information needs. Web search queries are distinctive in that they are often +> plain text or hypertext with optional search-directives (such as "and"/"or" +> with "-" to exclude). They vary greatly from standard query languages, which +> are governed by strict syntax rules as command languages with keyword or +> positional parameters. + +[Source](https://en.wikipedia.org/wiki/Web_search_query). + +## Faceted Search + +> Faceted search, also known as faceted navigation or faceted browsing, is a +> technique used by eCommerce brands to help users analyze, organize, and +> filter large sets of product inventory based on filters such as size, color, +> price, and brand. + +[Source](https://www.dynamicyield.com/glossary/faceted-search/). + +> Faceted search, also called faceted navigation or faceted browsing, is a +> technique for accessing information organized according to a faceted +> classification system, allowing users to explore a collection of information +> by applying multiple filters. A faceted classification system classifies each +> information element along multiple explicit dimensions, called facets, +> enabling the classifications to be accessed and ordered in multiple ways +> rather than in a single, pre-determined, taxonomic order. + +[Source](https://en.wikipedia.org/wiki/Faceted_search). + +## Search Filter + +> An extension of faceted search, a search filter is a specific product +> attribute a visitor can use to refine the search results of a particular +> category listing, e.g. by size, color, price, or brand. Multiple filters may +> be applied to take a broad range of products and refine them into a more +> narrow selection, allowing the end user to retrieve the most relevant search +> results based on the criteria they’ve selected. + +[Source](https://www.dynamicyield.com/glossary/search-filter/). + [↪ Next: Map of Repos](map-of-repos.md) From eebe193638619add66997178372af62ebe4c0458 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:29:37 +0000 Subject: [PATCH 058/150] More terms --- about/glossary-of-terms.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/about/glossary-of-terms.md b/about/glossary-of-terms.md index 1574bb1..c5d8788 100644 --- a/about/glossary-of-terms.md +++ b/about/glossary-of-terms.md @@ -67,4 +67,26 @@ According to Wikipedia, `Search Query`, in the context of the `web`, refers to: [Source](https://www.dynamicyield.com/glossary/search-filter/). +## Configuration Based Development + +> The difference between configuration-driven development and model-driven +> development is that the former is not restricted to the model of the code +> such as classes, fields, and relationships. Configuration-driven development +> (CCD) encompasses anything that can be configured within your application. +> For example, if your architecture dictates that particular business rules +> must be applied consistently across your application, you can use +> configuration files to configure and apply those rules. + +[Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). + +## Functional Logic Programming + +> Functional logic programming is the combination, in a single programming +> language, of the paradigms of functional programming (including higher-order +> programming) and logic programming (nondeterministic programming, +> unification). This style of programming is embodied by various programming +> languages, including Curry and Mercury. + +[Source](https://en.wikipedia.org/wiki/Functional_logic_programming). + [↪ Next: Map of Repos](map-of-repos.md) From dfe2d6db58f2de438916b0b3a748d06ca55650ec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:33:00 +0000 Subject: [PATCH 059/150] This is more accurate --- getting-started/setup.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/getting-started/setup.md b/getting-started/setup.md index aff2f54..76bd0dd 100644 --- a/getting-started/setup.md +++ b/getting-started/setup.md @@ -4,8 +4,8 @@ # Project Setup The whole Contexture Framework is available through NPM. It doesn't -have any extra dependency besides Node 9. In this quick guide, we'll -achieve the following goals: +have any extra dependency besides Node 9 and some NPM repositories. In +this quick guide, we'll achieve the following goals: - [Install Node 9 and NPM](./setup.md#installing-node-9-and-npm) - [Install Contexture](./setup.md#installing-contexture) From 2bda8c577cde2d6c18aaf55044603ae9764c6686 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:33:56 +0000 Subject: [PATCH 060/150] More accurate --- getting-started/setup.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/getting-started/setup.md b/getting-started/setup.md index 76bd0dd..7469027 100644 --- a/getting-started/setup.md +++ b/getting-started/setup.md @@ -16,8 +16,8 @@ this quick guide, we'll achieve the following goals: **Note:** as you progress through our documentation, you'll discover that in some cases, you will need only one or two of these -repositories. This page just provides the fastest start we can come up -with. +repositories. This page just provides the fastest way to start that we +couldcome up with. ## Installing Node 9 and NPM From 0ffecfeb14d61a256f8ff786e88cefb1d58d760a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:35:39 +0000 Subject: [PATCH 061/150] Typos --- getting-started/setup.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/getting-started/setup.md b/getting-started/setup.md index 7469027..dfb6054 100644 --- a/getting-started/setup.md +++ b/getting-started/setup.md @@ -45,28 +45,28 @@ project, then run: ## Installing Contexture Client -To install the `contexture-client` you can also run the followign +To install the `contexture-client` you can run the following command in your project's root folder: npm install --save contexture-client ## Installing Contexture ElasticSearch -To install the `contexture-elasticsearch` you can also run the followign -command in your project's root folder: +To install the `contexture-elasticsearch` you can also run the +following command in your project's root folder: npm install --save contexture-elasticsearch ## Installing Contexture Mongo -To install the `contexture-mongo` you can also run the followign +To install the `contexture-mongo` you can also run the following command in your project's root folder: npm install --save contexture-mongo ## Installing Contexture React -To install the `contexture-react` you can also run the followign +To install the `contexture-react` you can also run the following command in your project's root folder: npm install --save contexture-react From 5f6e1d23bed67ecba53bdf4728cb1a90b1e614b5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:52:01 +0000 Subject: [PATCH 062/150] Better separations --- getting-started/first-script.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/getting-started/first-script.md b/getting-started/first-script.md index ef27331..8f73de5 100644 --- a/getting-started/first-script.md +++ b/getting-started/first-script.md @@ -63,10 +63,17 @@ You can also try this same code in Runkit here: ## What it does +1. Requiring Contexture + ```javascript let contexture = require('contexture') ``` -We start by requiring `contexture`. +We start by requiring `contexture`. There's nothing much else to see +here. + +![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) + +2. Writing the Schemas ```javascript let schemas = { @@ -86,6 +93,8 @@ supported by a single provider `mongo`, from which we'll be looking for the `collectionName` collection. We'll learn more about the schemas later on, in [Querying → Schemas](../querying/schemas.md). +3. Writing the Contexture DSL + ```javascript let searchTree = { key: 'root', @@ -119,6 +128,8 @@ You can read more about these topics in the following links: - [Types and Type Components](../types/README.md). - [Mongo Example Types](../types/mongo-example-types.md). +3. Getting Results + ```javascript let result = await contexture({ schemas, @@ -167,6 +178,8 @@ You can read more about these concepts here: - [Contexture Providers](../under-the-hood/contexture-providers/README.md). - Contexture Core's [Default Export](../under-the-hood/contexture-core.md#default-export). +4. Exploring the Results + ```javascript console.log(result.children[1].context.response.results) From 20e92021c9540f3a41cba9cc3c49df6f9cc42a6c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 14:58:28 +0000 Subject: [PATCH 063/150] No async here --- getting-started/connecting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/getting-started/connecting.md b/getting-started/connecting.md index d4e2e6e..4841ce1 100644 --- a/getting-started/connecting.md +++ b/getting-started/connecting.md @@ -21,7 +21,7 @@ let elasticsearch = require('elasticsearch') let AgentKeepAlive = require('agentkeepalive') let elasticClient = null -let getClient = async () => { +let getClient = () => { if (elasticClient) return elasticClient elasticClient = elasticsearch.Client({ minSockets: 1, From 59bfd5279e874077a88e26868389d794eebe9d8f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 15:04:29 +0000 Subject: [PATCH 064/150] Another link regarding drivers --- getting-started/connecting.md | 1 + 1 file changed, 1 insertion(+) diff --git a/getting-started/connecting.md b/getting-started/connecting.md index 4841ce1..b96dcc6 100644 --- a/getting-started/connecting.md +++ b/getting-started/connecting.md @@ -313,6 +313,7 @@ You can read more about these topics in the following links: - [Contexture Core's Default Export](../under-the-hood/contexture-core.md#default-export). - [Contexture Providers in detail](under-the-hood/contexture-providers/README.md). +- [Connecting to MongoDB using the native MongoDB driver for NodeJS](http://mongodb.github.io/node-mongodb-native/api-articles/nodekoarticle1.html#getting-that-connection-to-the-database). This example and many other important details about `contexture-mongo` are accessible in the following links: From 4b7ebfdebc0d89b38272988bc3e01f1aa16245aa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 15:13:07 +0000 Subject: [PATCH 065/150] Simple search box page --- getting-started/simple-search-box.md | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 getting-started/simple-search-box.md diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md new file mode 100644 index 0000000..4e4046c --- /dev/null +++ b/getting-started/simple-search-box.md @@ -0,0 +1,6 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Connecting to Other Databases](connecting-other-databases.md) + +## Simple Search Box + +[↪ Next: Your First Filter](your-first-filter.md) From 3f853977e5d7664c9e5afb8b21950449382b41d2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 19:07:33 +0000 Subject: [PATCH 066/150] Simple search box first sketch --- getting-started/simple-search-box.md | 209 +++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 4e4046c..3dda4fc 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -3,4 +3,213 @@ ## Simple Search Box +Building a simple search box consists of a single input field tied to +any of the fields that any record might have in the specified database +or index. To be able to make this text input tied to the search +structure, we will need to bring `contexture-client` in. With that in +mind, and the knowledge we already have of contexture, we can define +the following tasks that we will need to: + +1. Create a New Contexture Search Function. +2. Create a Web Server With a Search Endpoint +3. Write a Search Tree. +4. Make the Search Tree Aware of Contexture. +5. Write a Text Input. + +Let's dive in. + +1. Creating a New Conteture Search Function + +Just as how we saw in the previous pages, creating a new Contexture +search function is about setting up the `contexture` package's default +epxort with `schemas` and `providers`. In this case, we'll use the +contexture-mongo approach in the following file (let's call it +`search.js`): + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient + +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} + +module.exports = {} + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + module.exports.search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +Note that we're now exporting the search function with +`module.exports.search = Contexture(...`. You can also note that we +specified the schema's name to be `collectionNameSchema`, and that any +search using this schema will be using MongoDb's `collectionName` +collection. + +2. Create a Web Server With a Search Endpoint + +For the current use case of providing a text input that will run the +searches, the previous code is going to live in the server. We'll need +to expose a `/search` endpoint so we can reach this function with the +client. For this purpose, we'll write a simple web server with +`express`, as follows: + +2.1. Installing the Dependencies + +For the following web server example, you'll need to install `express` +and `body-parser` at the root of your project, as follows: + + npm install --save express body-parser + +2.2. Writing the Web Server + +Once you have the dependencies installed, you can set up the server +with the following code: + +```javascript +let express = require('express') +let bodyParser = require('body-parser') +let search = require('./search') +let app = express() + +// create application/json parser +lete jsonParser = bodyParser.json() + +app.post('/search', jsonParser, (req, res) => { + if (!req.body || !req.body.search) return res.sendStatus(400) + search(req.body.search).then((err, result) => { + if (err) return res.send(401, err) + res.send(200, result) + }) +}) + +app.listen(3000) +``` + +You can read more about these topics in the following links: + +- [Express Documentation](https://expressjs.com/en/api.html). +- [body-parser repository](https://github.com/expressjs/body-parser). + +3. Writing a Search Tree + +Once we have the contexture DSL processor available through an +endpoint of a web server, we can follow up with the structure of the +search interface that we will be writing. We'll conceptualize this by +writing the Contexture DSL itself. + +Let's use the same `sarchTree` that we used in [our frist +script](first-script.md): + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} +``` + +4. Make the Search Tree Aware of Contexture + +Having a search tree, we will need `contexture-client` to make it +smart enough for the user interface. Let's make sure we have it +available in our project with: + + npm install --save contexture-client + +We will also use MobX to make it easier to notify our component that +the state has changed. For that purpose, we will need to install +`mobx` using NPM: + + npm install --save mobx + +Besides the `mobx` dependency, `contexture-client` already provides a +mobx adapter that we can use out of the box. So, let's first of all +prepare our `contexture-client` to work well with `mobx`: + +```javascript +let ContextureClient = require('contexture-client') +let ContextureMobx = require('contexture-react/dist/utils/contexture-mobx') + +let types = ContextureClient.exampleTypes +let service = async search => ({ + data: await postData('/sarch', { search }) +}) + +let Contexture = ContextureMobx({ + types, + service, +}) +``` + +With `contexture-client` already prepared with `mobx` and our service +function, which will send the `searchTree` we previously defined to +the DSL processor, we can wrap the search tree into a smart object +that will later react to both the user input, and the search results. + +```javascript +let contextureSearchTree = Contexture(searchTree) +``` + +5. Writing a Text Input + +Having the search tree ready allows us to write a `mobx` observer +component that will receive the tree and react to the result changes +immediately. Here's an example: + +``` +let { observer } = require('mobx') +let SearchQuery = observer(({ tree }) => + { + tree.getNode(['root', 'namequery']).value = e.target.value + }} + /> + +let SearchResults = observer(({ tree }) => ( +
+ {JSON.stringify(tree.getNode(['root', 'results']).context.response.results)} +
+)) +``` + +Which would be rendered like: + +```javascript + + +``` + +This will generate a text input that will trigger a new search every +time the contents of the input change. This new search will get the +results and show the results in a JSON form (because of the +`JSON.stringify` part). Ideally, you will not render them in a JSON +form, but render them using a list component or table. + [↪ Next: Your First Filter](your-first-filter.md) From c13821d181d11161165b6b2c71563fd677ac25e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 19:10:35 +0000 Subject: [PATCH 067/150] lil bit better --- getting-started/simple-search-box.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 3dda4fc..5198348 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -8,7 +8,7 @@ any of the fields that any record might have in the specified database or index. To be able to make this text input tied to the search structure, we will need to bring `contexture-client` in. With that in mind, and the knowledge we already have of contexture, we can define -the following tasks that we will need to: +the following tasks: 1. Create a New Contexture Search Function. 2. Create a Web Server With a Search Endpoint @@ -22,7 +22,7 @@ Let's dive in. Just as how we saw in the previous pages, creating a new Contexture search function is about setting up the `contexture` package's default -epxort with `schemas` and `providers`. In this case, we'll use the +export with `schemas` and `providers`. In this case, we'll use the contexture-mongo approach in the following file (let's call it `search.js`): From 9f82c48fdadb2bd92acdedd334a0b38cadb74557 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 19:12:01 +0000 Subject: [PATCH 068/150] Less as follows --- getting-started/simple-search-box.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 5198348..9ea4ac3 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -67,7 +67,7 @@ For the current use case of providing a text input that will run the searches, the previous code is going to live in the server. We'll need to expose a `/search` endpoint so we can reach this function with the client. For this purpose, we'll write a simple web server with -`express`, as follows: +`express`. 2.1. Installing the Dependencies From 8c48c88b2cc613507b6391290ce2852621fad3be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 19:21:39 +0000 Subject: [PATCH 069/150] Better Headers --- getting-started/simple-search-box.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 9ea4ac3..d803262 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -18,7 +18,7 @@ the following tasks: Let's dive in. -1. Creating a New Conteture Search Function +### 1. Creating a New Conteture Search Function Just as how we saw in the previous pages, creating a new Contexture search function is about setting up the `contexture` package's default @@ -61,7 +61,7 @@ specified the schema's name to be `collectionNameSchema`, and that any search using this schema will be using MongoDb's `collectionName` collection. -2. Create a Web Server With a Search Endpoint +### 2. Create a Web Server With a Search Endpoint For the current use case of providing a text input that will run the searches, the previous code is going to live in the server. We'll need @@ -69,14 +69,14 @@ to expose a `/search` endpoint so we can reach this function with the client. For this purpose, we'll write a simple web server with `express`. -2.1. Installing the Dependencies +#### 2.1. Installing the Dependencies For the following web server example, you'll need to install `express` and `body-parser` at the root of your project, as follows: npm install --save express body-parser -2.2. Writing the Web Server +#### 2.2. Writing the Web Server Once you have the dependencies installed, you can set up the server with the following code: @@ -106,7 +106,7 @@ You can read more about these topics in the following links: - [Express Documentation](https://expressjs.com/en/api.html). - [body-parser repository](https://github.com/expressjs/body-parser). -3. Writing a Search Tree +### 3. Writing a Search Tree Once we have the contexture DSL processor available through an endpoint of a web server, we can follow up with the structure of the @@ -134,7 +134,7 @@ let searchTree = { } ``` -4. Make the Search Tree Aware of Contexture +### 4. Make the Search Tree Aware of Contexture Having a search tree, we will need `contexture-client` to make it smart enough for the user interface. Let's make sure we have it @@ -176,7 +176,7 @@ that will later react to both the user input, and the search results. let contextureSearchTree = Contexture(searchTree) ``` -5. Writing a Text Input +### 5. Writing a Text Input Having the search tree ready allows us to write a `mobx` observer component that will receive the tree and react to the result changes From 3a167124dd701f4d41f3f20bff322df982cabd5a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 19:43:08 +0000 Subject: [PATCH 070/150] More information --- getting-started/simple-search-box.md | 45 +++++++++++++++++++--------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index d803262..6e8eee3 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -63,16 +63,16 @@ collection. ### 2. Create a Web Server With a Search Endpoint -For the current use case of providing a text input that will run the -searches, the previous code is going to live in the server. We'll need -to expose a `/search` endpoint so we can reach this function with the -client. For this purpose, we'll write a simple web server with -`express`. +If we want to separate the direct access to the database from the +client, the previous code should live in the server. Our next step is +to expose a `/search` endpoint so our future client can reach this +function. In this section, we'll write a simple web server with +`express` to satisfy our needs. #### 2.1. Installing the Dependencies -For the following web server example, you'll need to install `express` -and `body-parser` at the root of your project, as follows: +You'll need to install `express` and `body-parser` at the root of your +project, as follows: npm install --save express body-parser @@ -108,10 +108,9 @@ You can read more about these topics in the following links: ### 3. Writing a Search Tree -Once we have the contexture DSL processor available through an -endpoint of a web server, we can follow up with the structure of the -search interface that we will be writing. We'll conceptualize this by -writing the Contexture DSL itself. +Having the DSL processor available through a web server endpoint, we +can follow up with the structure of the search interface itself. We'll +conceptualize this by writing the Contexture DSL itself. Let's use the same `sarchTree` that we used in [our frist script](first-script.md): @@ -134,6 +133,10 @@ let searchTree = { } ``` +Keep in mind that we'll be using `collectionNameSchema` since we +already defined a schema with that name on the server's `search.js` +file. + ### 4. Make the Search Tree Aware of Contexture Having a search tree, we will need `contexture-client` to make it @@ -148,9 +151,10 @@ the state has changed. For that purpose, we will need to install npm install --save mobx -Besides the `mobx` dependency, `contexture-client` already provides a -mobx adapter that we can use out of the box. So, let's first of all -prepare our `contexture-client` to work well with `mobx`: +One of the advantages of the `mobx` dependency is that our +`contexture-client` already provides a mobx adapter that we can use +out of the box. So, let's first of all prepare our `contexture-client` +to work well with `mobx`: ```javascript let ContextureClient = require('contexture-client') @@ -176,6 +180,12 @@ that will later react to both the user input, and the search results. let contextureSearchTree = Contexture(searchTree) ``` +You can read more about these topics here: + +- [MobX](https://mobx.js.org/). +- [MobX Observers](https://mobx.js.org/refguide/observer-component.html). +- [Managing the Contexture Search Tree State with MobX](../managing-state/mobx.md). + ### 5. Writing a Text Input Having the search tree ready allows us to write a `mobx` observer @@ -212,4 +222,11 @@ results and show the results in a JSON form (because of the `JSON.stringify` part). Ideally, you will not render them in a JSON form, but render them using a list component or table. +More information here: + +- [Interactive Queries](../interactive-queries/README.md). +- [Available React Components for our Available Types](../types/react-components.md). +- [Managing State](../managing-state/README.md). +- [Recommendations](../recommendations/README.md). + [↪ Next: Your First Filter](your-first-filter.md) From 1b182e4a4c2466156daf58b2e53b1361b68b81f3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Thu, 31 May 2018 21:41:24 +0000 Subject: [PATCH 071/150] Discover the database and your first filter --- getting-started/discovering-the-database.md | 121 ++++++++++++++++++++ getting-started/your-first-filter.md | 74 ++++++++++++ 2 files changed, 195 insertions(+) create mode 100644 getting-started/discovering-the-database.md create mode 100644 getting-started/your-first-filter.md diff --git a/getting-started/discovering-the-database.md b/getting-started/discovering-the-database.md new file mode 100644 index 0000000..aa4cde7 --- /dev/null +++ b/getting-started/discovering-the-database.md @@ -0,0 +1,121 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Your First Filter](your-first-filter.md) + +## Discovering The Database + +One of the great things about Contexture is that it can be used to +discover the database. In this page, we'll see how to write a filter +that not only allows the user to refine their search, but that also +shows previously unknown information about the database. + +### The Facet Filter + +The `facet` type is just like any other type in the tree. It requires +a unique key and some other properties. In contrast to previously seen +types, such as `text` and `number`, facet doesn't require for specific +values, but instead it asks for two properties: `field`, and +optionally `size`. This type is increadibly useful because besides +filtering the results based on the `value` (or `values`) the user +might have chosen, it retrieves from the database a list of available +values! The `facet` type can be very well represented as a list of +checkboxes, ready for users to pick for one or more values. Let's see +how we can use it. + +### Adding the Facet Filter to the Tree + +To add the `facet` filter to the tree, we simply add it to the +structure we already had. For example: + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'numberrange', + type: 'number', + field: 'age', + min: 0, + max: 100 + }, { + key: 'cityfacet', + type: 'facet', + field: 'city', + value: 'Orlando', // Optional + // We could have instead `values` with ['Orlando', 'Miami'] + }, { + key: 'results', + type: 'results' + }] +} +``` + +Once we have it in the tree, we can simply create another component to +allow users to set a value to this type. In this case, however, we +will be retrieving the available values from a field that is +automatically inserted to this part of the tree. Let's make sure we +have the data before we render the components. + + +### Fetching the Available Options + +Having the facet node already added to the tree, it's only matter of +running a first search before we render the components to actually get +the possible results from the `facet` type. We can achieve this by +calling to `contexture-client`'s `refresh` function: + +```javascript +// This would go after: +// let contextureSearchTree = Contexture(searchTree) +contextureSearchTree.refresh(['root']) +``` + +Now, the available options will be ready to be used at: +`contextureSearchTree.getNode(['root', 'cityfacet']).context.options`. +So we can write our facet component this way: + +```javascript +let toggleValue = (array, value) => { + if (array.indexOf(value) > -1) { + let copy = array.slice() // Making sure it's not a MobX Array + copy.splice(array.indexOf(value), 1) // removing value from the array + array.replace(copy) // MobX Arrays have this method to replace the array's inner values + } else { + array.push(value) + } +} +let RangeComponent = observer(({ tree }) => ( +
+ Select One or More: + {tree.getNode(['root', 'cityfacet']).context.options.map(option => ( +
+ -1} + onChange={() => toggleValue(tree.getNode(['root', 'cityfacet']).values, option.value)} + /> + +
+ ))} +
+)) +``` + +Including this component in our search interface will show the current +available cities for the search results we have so far, and will allow +users to filter the search even more by letting them pick any (or many) +of the available cities. + +You can read more about our available types here: + +- [Types and Type Components](../types/README.md) + +[↪ Next: IMDB Index](imdb-the-database.md) diff --git a/getting-started/your-first-filter.md b/getting-started/your-first-filter.md new file mode 100644 index 0000000..88d0278 --- /dev/null +++ b/getting-started/your-first-filter.md @@ -0,0 +1,74 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Simple Search Box](simple-search-box.md) + +## Your First Filter + +In the previous page, we wrote a simple search user interface where a +single input would retrieve results. Now, we will add a simple filter +that will allow us to get results within a given range. + +### Adding the Filter to the Tree + +The way we will add the filter is by adding a node to the tree. This +node is going to have a type of `number`, which asks for a `field`, a +`min` and a `max` values. + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'numberrange', + type: 'number', + field: 'age', + min: 0, + max: 100 + }, { + key: 'results', + type: 'results' + }] +} +``` + +With the `numberrange` node added, we can create another component +with two inputs that will allow us to filter results with the range +that the user specifies: + +```javascript +let RangeComponent = observer(({ tree }) => ( +
+ Minimum: + { + tree.getNode(['root', 'numberrange']).min = e.target.min + }} + /> +
+ Maximum: + { + tree.getNode(['root', 'numberrange']).max = e.target.max + }} + /> +
+)) +``` + +And that's it! Rendering this component will make our search aware of +any change the user might desire on the minimum and maximum ages they +might want to use to filter the available results. + +You can read more about our available types here: + +- [Types and Type Components](../types/README.md) + +[↪ Next: Discovering the Database](discovering-the-database.md) From 380ccf3542d72eb143a1a27f03d73b7693e82737 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 1 Jun 2018 13:55:14 +0000 Subject: [PATCH 072/150] More references --- getting-started/simple-search-box.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 6e8eee3..d189d89 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -137,6 +137,9 @@ Keep in mind that we'll be using `collectionNameSchema` since we already defined a schema with that name on the server's `search.js` file. +You can read more about writing Contexture DSL queries in our +[Querying](../querying/README.md) documentation. + ### 4. Make the Search Tree Aware of Contexture Having a search tree, we will need `contexture-client` to make it From c28c3a08e12f98d8f5ec8bd74eb159762c40b2f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 1 Jun 2018 15:24:53 +0000 Subject: [PATCH 073/150] imdb example --- README.md | 2 +- getting-started/README.md | 2 +- getting-started/discovering-the-database.md | 8 +-- getting-started/imdb-example.md | 73 +++++++++++++++++++++ getting-started/simple-search-box.md | 19 +++--- 5 files changed, 89 insertions(+), 15 deletions(-) create mode 100644 getting-started/imdb-example.md diff --git a/README.md b/README.md index 6967685..eaa31d6 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Simple Search Box](getting-started/simple-search-box.md) - [Your First Filter](getting-started/your-first-filter.md) - [Discovering the Database](getting-started/discovering-the-database.md) - - [IMDB Index](getting-started/IMDB-example.md) + - [IMDB Index](getting-started/imdb-example.md) - [Querying](querying/README.md) - [Contexture Core](querying/contexture-core.md) - [Contexture DSL](querying/contexture-dsl.md) diff --git a/getting-started/README.md b/getting-started/README.md index d326132..47dbb00 100644 --- a/getting-started/README.md +++ b/getting-started/README.md @@ -18,4 +18,4 @@ Table of Contents: - [Simple Search Box](simple-search-box.md) - [Your First Filter](your-first-filter.md) - [Discovering the Database](discovering-the-database.md) -- [IMDB Index](IMDB-example.md) +- [IMDB Index](imdb-example.md) diff --git a/getting-started/discovering-the-database.md b/getting-started/discovering-the-database.md index aa4cde7..e151b88 100644 --- a/getting-started/discovering-the-database.md +++ b/getting-started/discovering-the-database.md @@ -6,13 +6,14 @@ One of the great things about Contexture is that it can be used to discover the database. In this page, we'll see how to write a filter that not only allows the user to refine their search, but that also -shows previously unknown information about the database. +shows information about our data that might not be obvious by looking +at the results directly. ### The Facet Filter The `facet` type is just like any other type in the tree. It requires a unique key and some other properties. In contrast to previously seen -types, such as `text` and `number`, facet doesn't require for specific +types, such as `text` and `number`, facet doesn't require specific values, but instead it asks for two properties: `field`, and optionally `size`. This type is increadibly useful because besides filtering the results based on the `value` (or `values`) the user @@ -62,7 +63,6 @@ will be retrieving the available values from a field that is automatically inserted to this part of the tree. Let's make sure we have the data before we render the components. - ### Fetching the Available Options Having the facet node already added to the tree, it's only matter of @@ -118,4 +118,4 @@ You can read more about our available types here: - [Types and Type Components](../types/README.md) -[↪ Next: IMDB Index](imdb-the-database.md) +[↪ Next: IMDB Index](imdb-example.md) diff --git a/getting-started/imdb-example.md b/getting-started/imdb-example.md new file mode 100644 index 0000000..bf64757 --- /dev/null +++ b/getting-started/imdb-example.md @@ -0,0 +1,73 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Your First Filter](your-first-filter.md) + +## IMDB Index + +In our Contexture React repository, we've created a Storybook live +example that uses Contexture (with no server whatsoever, everything in +the client) to query and discover a public ElasticSearch index with +data similar to the data that IMDB might contain. + +- [Contexture React Repository](https://github.com/smartprocure/contexture-react). +- More information about [Storybook](https://github.com/storybooks/storybook). +- [The IMDB Live Index Explorer with Contexture](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). +- [The IMDB Live Index Explorer Source Code](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). + +Let's see how we can use this tool. + +### Picking a Field + +The tool initially shows you a button named `Pick a Field`: + +![](https://i.imgur.com/CWz4Moy.png) + +This button will allow you to select any field on which you might want +to run a search. If you click it, you'll see the following fields: + +![](https://i.imgur.com/RR6vMP1.png) + +If we pick `Actors`, for example, the search will trigger and results +will appear: + +![](https://i.imgur.com/kep1LIo.png) + +Now, let's click on `Select Type` and click on `Facet`. A moment +afterwards, we will get a list of checkboxes with the most common +actors among the results: + +![](https://i.imgur.com/2Gq5FEh.png) + +We can see that the actor that has made the most movies (among the +ones idexed) is Naveen Andrews. We can also see the number of movies +this and other actors have made, and the total actors in the database +(of which we can only see 10 in this screenshot). Under this list of +actors, you'll see a text saying `View More`. If you click it, the +next 10 actors will appear (in order). + +At the bottom of the page, you'll see a button labeled `Add Filter`. +Clicking it will add another interactive component to the website, +which will say `Click to add AND`. + +![](https://i.imgur.com/sRHN04n.png) + +Clicking the `Click to add AND` button will show us again a `Pick a +Field` button: + +![](https://i.imgur.com/oA4LOEK.png) + +So we can start again. Let's say we pick the `Genre` field, and we +click for the `Facet` type again. A bit later, the list of ordered +genres will appear: + +![](https://i.imgur.com/nfbVlQ2.png) + +And that's it! We're discovering the database with a very simple and +unpolished interface. By this point you might be curious on what +components we're using to do all this magic. There's really no trick, +just follow us through the tutorial. You can also skip some steps if +you're particularly interested in: + +- [Contexture React Repository](https://github.com/smartprocure/contexture-react). +- [Available React Components for Types](../types/react-components.md). + +[↪ Next: Querying](../querying/README.md) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index d189d89..6bf45d6 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -154,10 +154,11 @@ the state has changed. For that purpose, we will need to install npm install --save mobx -One of the advantages of the `mobx` dependency is that our -`contexture-client` already provides a mobx adapter that we can use -out of the box. So, let's first of all prepare our `contexture-client` -to work well with `mobx`: +Since we're heavy users of MobX, `contexture-client` already +provides an adapter that we can use out of the box. Knowing this, +let's prepare our `contexture-client` to work well with +`mobx`, as follows: + ```javascript let ContextureClient = require('contexture-client') @@ -174,10 +175,10 @@ let Contexture = ContextureMobx({ }) ``` -With `contexture-client` already prepared with `mobx` and our service -function, which will send the `searchTree` we previously defined to -the DSL processor, we can wrap the search tree into a smart object -that will later react to both the user input, and the search results. +Note that our service function will be the one responsible for sending +the `searchTree` we previously defined to the DSL processor, we can +wrap the search tree into a smart object that will later react to both +the user input, and the search results. ```javascript let contextureSearchTree = Contexture(searchTree) @@ -212,7 +213,7 @@ let SearchResults = observer(({ tree }) => ( )) ``` -Which would be rendered like: +Which we would render this way: ```javascript From e18df96c8c053f7050bfca0b5aa65e55e3340095 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 1 Jun 2018 15:28:59 +0000 Subject: [PATCH 074/150] Typo --- getting-started/discovering-the-database.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/getting-started/discovering-the-database.md b/getting-started/discovering-the-database.md index e151b88..0cf2629 100644 --- a/getting-started/discovering-the-database.md +++ b/getting-started/discovering-the-database.md @@ -15,7 +15,7 @@ The `facet` type is just like any other type in the tree. It requires a unique key and some other properties. In contrast to previously seen types, such as `text` and `number`, facet doesn't require specific values, but instead it asks for two properties: `field`, and -optionally `size`. This type is increadibly useful because besides +optionally `size`. This type is incredibly useful because besides filtering the results based on the `value` (or `values`) the user might have chosen, it retrieves from the database a list of available values! The `facet` type can be very well represented as a list of From c0a94f26d26aa1d44b877597522d2f3ca4cf6c7b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 1 Jun 2018 21:36:36 +0000 Subject: [PATCH 075/150] Lots of stuff --- README.md | 21 +- getting-started/simple-search-box.md | 4 + interactive-queries/README.md | 7 - managing-state/README.md | 1 + querying/README.md | 10 +- querying/contexture-dsl.md | 58 ++++ querying/interactive-queries/README.md | 13 + .../interactive-queries/contexture-client.md | 179 +++++++++++ querying/interactive-queries/reactors.md | 44 +++ types/README.md | 3 +- types/diy-types.md | 10 +- types/elasticsearch-example-types.md | 288 +++++++++++++++++- under-the-hood/README.md | 6 +- 13 files changed, 615 insertions(+), 29 deletions(-) delete mode 100644 interactive-queries/README.md create mode 100644 managing-state/README.md create mode 100644 querying/contexture-dsl.md create mode 100644 querying/interactive-queries/README.md create mode 100644 querying/interactive-queries/contexture-client.md create mode 100644 querying/interactive-queries/reactors.md diff --git a/README.md b/README.md index eaa31d6..f57dd25 100644 --- a/README.md +++ b/README.md @@ -29,17 +29,18 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Discovering the Database](getting-started/discovering-the-database.md) - [IMDB Index](getting-started/imdb-example.md) - [Querying](querying/README.md) - - [Contexture Core](querying/contexture-core.md) - [Contexture DSL](querying/contexture-dsl.md) - - [Available Providers](querying/available-providers.md) - - [Schemas](querying/schemas.md) -- [Interactive Queries](interactive-queries/README.md) - - [Contexture Client](interactive-queries/contexture-client.md) - - [Context Tree](interactive-queries/contexture-client.md#context-tree) - - [Introduction to Reactors](interactive-queries/reactors.md) + - [Interactive Queries](querying/interactive-queries/README.md) + - [Contexture Client](querying/interactive-queries/contexture-client.md) + - [Initializing the Contexture Client](querying/interactive-queries/contexture-client.md#initializing-the-contexture-client) + - [Context Tree](querying/interactive-queries/contexture-client.md#context-tree) + - [Tree and getNode](contexture-client.md#tree-and-getnode) + - [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) + - [Introduction to Reactors](querying/interactive-queries/reactors.md) - [Types and Type Components](types/README.md) - [DIY Types](types/diy-types.md) - - [How to Write a Type](types/diy-types.md#how-to-wite-a-type) + - [How to Write a Provider Type](types/diy-types.md#how-to-wite-a-provider-type) + - [How to Write a Client Type](types/diy-types.md#how-to-wite-a-client-type) - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) - [The Example Types](types/diy-types.md#the-example-types) - [ElasticSearch Example Types](types/elasticsearch-example-types.md) @@ -47,6 +48,7 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Available React Components for Types](types/react-components.md) - [Other Components](other-components/README.md) - [Managing State](managing-state/README.md) + - [DIY State Management](managing-state/diy.md) - [MobX](managing-state/mobx.md) - [Redux (Coming Soon)](managing-state/redux.md) - [Theming](theming/README.md) @@ -73,8 +75,9 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Contexture Mongo](under-the-hood/contexture-providers/) - [Building Your Own Provider](under-the-hood/contexture-providers/building-your-own-provider.md) - [Contexture Client](under-the-hood/contexture-client.md) + - [State Properties](under-the-hood/contexture-client.md#state-properties) - [Lenses](under-the-hood/contexture-client.md#lenses) - - [Reactors in detail](under-the-hood/contexture-client.md#reactors-in-detail) + - [Reactors in Detail](under-the-hood/contexture-client.md#reactors-in-detail) - [Serialization](under-the-hood/contexture-client.md#serialization) - [Traversals](under-the-hood/contexture-client.md#traversals) - [Contexture React](under-the-hood/contexture-react.md) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 6bf45d6..cb7f93b 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -189,6 +189,10 @@ You can read more about these topics here: - [MobX](https://mobx.js.org/). - [MobX Observers](https://mobx.js.org/refguide/observer-component.html). - [Managing the Contexture Search Tree State with MobX](../managing-state/mobx.md). +- [Managing the Contexture Search Tree State with MobX](../managing-state/mobx.md). +- [Introduction to the Contexture Client for Easy Querying](../querying/interactive-queries/contexture-client.md). +- [Introduction to the Contexture Client for Easy Querying](../querying/interactive-queries/contexture-client.md). +- [Contexture Client in Detail](../under-the-hood/contexture-client.md). ### 5. Writing a Text Input diff --git a/interactive-queries/README.md b/interactive-queries/README.md deleted file mode 100644 index 5f5b4b6..0000000 --- a/interactive-queries/README.md +++ /dev/null @@ -1,7 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - -# Interactive Queries - -Table of Contents: -- [Contexture Client](contexture-client.md) -- [Introduction to Reactors](reactors.md) diff --git a/managing-state/README.md b/managing-state/README.md new file mode 100644 index 0000000..8b8ceb8 --- /dev/null +++ b/managing-state/README.md @@ -0,0 +1 @@ +# Managing State diff --git a/querying/README.md b/querying/README.md index 726a838..51d9b0c 100644 --- a/querying/README.md +++ b/querying/README.md @@ -1,9 +1,11 @@ [↩ Parent: Table of Contents](../README.md) - # Querying +# Querying Table of Contents: -- [Contexture Core](contexture-core.md) - [Contexture DSL](contexture-dsl.md) -- [Available providers](available-providers.md) -- [Schemas](schemas.md) +- [Interactive Queries](interactive-queries/README.md) + - [Contexture Client](interactive-queries/contexture-client.md) + - [Initializing the Contexture Client](interactive-queries/contexture-client.md#initializing-the-contexture-client) + - [Context Tree](interactive-queries/contexture-client.md#context-tree) + - [Introduction to Reactors](interactive-queries/reactors.md) diff --git a/querying/contexture-dsl.md b/querying/contexture-dsl.md new file mode 100644 index 0000000..92231f2 --- /dev/null +++ b/querying/contexture-dsl.md @@ -0,0 +1,58 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Querying](README.md) + +## Contexture DSL + +The Contexture DSL is a JavaScript object structure, or JSON +structure, that is composed of nested nodes that are equal at all +levels. Each node has a `key`, a `type` and many other optional +properties. The first node is called the root and is a node of type +`group`. Any node of type `group` can have many children. Each +children can be a node of any type. If any children is another group, +this children will probably have one or more other children nodes of +any type, and so on. + +Let's begin talking about the root. + +### The Root + +The root of a Contexture Tree is a node of type `group`. Group types +are required to have a `key`, the `type: "group"` property, and an +array of children `children: [/* nodes */]`. Root groups also need an +extra property: then ame of the schema. In summary, this is how a root +node should look: + +```javascript +let searchTree = { + key: 'myRootNode', + type: 'group', + schema: 'mySchemaName', + children: [ + // Other nodes + ] +} +``` + +### The Children + +Each children will be an individual node. Each node will have at least +a unique `key` (unique per tree, but they can appear again in other +trees), and a `type`. Some types might require more properties. You +can learn more about our types at: + +- [Types and Type Components](types/README.md) + +That's really all that it is for our DSL. The magic happens in the +types and the providers. As long as your types are defined properly, +and your providers build the correct queries and write them back in +the tree, Contexture will work and you'll be able to use the +Contexture DSL to build search interfaces of any complexity. + +**Note:** Many of the types have some common properties, like `field` +or `values`. The only reason these properties are common is because +they've made sense to be common for each one of our specific types, +not because they have something to share between types. Each type has +it's own properties and rules, so you should treat them as independent +structures. + +[↪ Next: Interactive Queries](interactive-queries/README.md) diff --git a/querying/interactive-queries/README.md b/querying/interactive-queries/README.md new file mode 100644 index 0000000..e94fddf --- /dev/null +++ b/querying/interactive-queries/README.md @@ -0,0 +1,13 @@ +[↩ Parent: Querying](../README.md) + +# Interactive Queries + +Table of Contents: +- [Contexture Client](contexture-client.md) + - [Initializing the Contexture Client](contexture-client.md#initializing-the-contexture-client) + - [Contexture Client Types](contexture-client.md#contexture-client-types) + - [Contexture Client Service](contexture-client.md#contexture-client-service) + - [Context Tree](contexture-client.md#context-tree) + - [Tree and getNode](contexture-client.md#tree-and-getnode) + - [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) +- [Introduction to Reactors](reactors.md) diff --git a/querying/interactive-queries/contexture-client.md b/querying/interactive-queries/contexture-client.md new file mode 100644 index 0000000..a0e0fd1 --- /dev/null +++ b/querying/interactive-queries/contexture-client.md @@ -0,0 +1,179 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Interactive Queries](README.md) + +## Contexture Client + +Having a Contexture DSL by itself won't work since it's only a +language and not an interactive or working program in any sense. We +need something to automatically send this query to our DSL processor +(the main `contexture` repository). For this purpose, we provide the +`contexture-client`. + +The Contexture Client is responsible for keeping track of the changes +that happen on the Contexture DSL, either by the user or by the +Contexture core. The nodes in the Contexture DSL host values used for +filtering the results, but also hold other properties, such as the +search results, and a `_meta` object that you can use for debugging +purposes. In this page, we will sumarize how the Contexture Client +works with the Contexture DSL to provide real-time interactive +searches. + +### Initializing the Contexture Client + +The Contexture Client has a default export that needs to be called +first with some configuration properties before being able to process +any search. This is how you would normally initialize it: + +```javascript +let ContextureClient = require('contexture-client') +let Contexture = ContextureClient({ + service, + types, +}) +``` + +When Contexture Client determines the search query is ready to be +executed, it will call the `service` function provided with the whole +Contexture DSL. It expects to receive another full DSL that it parses +to then change the nodes again. As previously mentioned, an example +`service` would be: + +```javascript +let service = async search => ({ + data: await postData('/sarch', { search }) +}) +``` + +The `types` property is an object where each key is a type's name. +These type definitions are different than the ones from the servers +and allow our Contexture Client to do three things: + +- To know how to validate a node. +- To know what happens to a specific node (and it's suroundings) when + it changes. +- To know how to complemente any missing value on a specific node + (default values, so you can omit properties when you write each of + the nodes). + +Each one of the types will have the following properties: + +| Property Name | Type | Description | +| --- | --- | --- | +| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | +| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | +| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | + +For more information, check the following links: + +- [How to make your own types (client and provider)](../../types/diy-types.md). +- [Specifically on how to make client types](../../types/diy-types.md#how-to-wite-a-client-type). + +### Context Tree + +Once you have Contexture Client initialized, we will be able to pass +the Contexture DSL into it. This will generate something we call the +`Context Tree`, which is an object with the following properties: + +| Name | Signature | Description | +| ---- | --------- | ----------- | +| add | `async (path, newNode) -> await searchCompleted` | Adds a node to the tree as a child of the specified path. You can await this for when updates settle and relevant searches are completed. | +| remove | `async path -> await searchCompleted` | Removes a node at the specified path. You can await this for when updates settle and relevant searches are completed. | +| mutate | `async (path, deltas) -> await searchCompleted` | Mutates the node at the given path with the new values. You can await this for when updates settle and relevant searches are completed. | +| getNode | `[path] -> node` | Lookup a node by a path (array of keys). | +| tree | tree | A reference to the internal tree. If you mutate this, you should dispatch an appropriate event. | + +You can read more about the instantiated client here: [contexture-client Run Time](https://github.com/smartprocure/contexture-client#run-time). + +You can also read more about the Contexture Client in our detailed +documentation: [Under the Hood: Contexture Client](../../under-the-hood/contexture-client.md). + +### Tree and getNode + +Context Trees have two properties for nagivating through the +Contexture DSL: `tree` and `getNode`. The `tree` property will have +the DSL as it was received, plus the default properties set by each +one of the client types, and some state properties. On the other hand, +`getNode` is a quick way to access the tree by sending only the keys +of each node. For example, let's say we have the following DSL: + +```javascript +let searchTree = ContextureClient({ + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }] +}) +``` + +We are able to access the `namequery` node by simply calling: + +```javascript +let nameQueryNode = searchTree.getNode(['root', 'namequery']) +``` + +### Mutate, Add and Remove + +Once you have a Context Tree ready, you will trigger searches +automatically by invoking one of the following functions: `add`, +`remove`, `mutate`. More specifically: + +- If you want to run a search after changing some of the properties of + a specific node, you would call `mutate`. +- If you want to add a children to any given `group`, even the root + group, you would call `add`. +- If you want to remove a children to any given `group`, even the root + group, you would call `remove`. + +Let's see some examples: + +1. Mutate + +Mutate allows us to change properties on nodes and trigger search +afterwards. Here is an example: + +```javascript +await searchTree.mutate(['root', 'namequery'], { + value: 'new value' +}) +``` + +This will change the tree and produce a new set of results. If you're +wondering how to keep track of these changes, the simplest way to do +it is by using our MobX adapter, as shown in our [simple search +box](../../getting-started/simple-search-box.md) tutorial, or in +greater detail in our [Managing State](managing-state/README.md) docs. + +2. Add + +Having the previous search tree, we can add a children by doing: + +```javascript +await searchTree.add(['root'], { + key: 'results', + type: 'results' +}) +``` + +3. Remove + +We can remove the `results` node we just added by doing: + +```javascript +await searchTree.add(['root', 'results']) +``` + +Calling `mutate`, `add` or `remove` will trigger events not only for +the node that these functions are targetting, but also for nearby +nodes, depending on the types. + +Up next, we'll dig a bit into what are the client side reactors (the +rules that Contexture Client follows to know what other nodes are +relevant for each update). + +[↪ Next: Reactors](reactors.md) diff --git a/querying/interactive-queries/reactors.md b/querying/interactive-queries/reactors.md new file mode 100644 index 0000000..cd8b84c --- /dev/null +++ b/querying/interactive-queries/reactors.md @@ -0,0 +1,44 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Contexture Client](contexture-client.md) + +## Reactors + +When any node changes, depending on the type, it might be reasonable +to re-trigger a search call for other nodes. We call this process the +selection of `reactors`, where the possible reactors are only three: +`self`, `others` and `all`. + +Reactors should be specified in the Client Types, where each type will +have a specific reactor for each one of it's properties. For example +(taken from our client side [example +types](https://github.com/smartprocure/contexture-client/blob/master/src/exampleTypes.js)): + +```javascript +facet: { + reactors: { + values: 'others', + mode: 'others', + size: 'self', + optionsFilter: 'self', + sort: 'self', + } +} +``` + +The client side type defined above will be effective for any node with +type `facet`, where the properties `values` and `mode` will affect +only all the other nodes (and not itself), and the properties `size`, +`optionsFilter` and `sort` will affect only the specific `facet` node +and no other node. + +The one remaining reactor that isn't covered by that example is the +`all` reactor. The difference between `others` and `all` is that +`others` excludes the node where the change is happening, and `all` +includes all other nodes and the node where the change is happening +(effectively combining `self` and `others`). + +You can also read more about reactors in the Contexture Client in our +detailed documentation: [Under the Hood: Contexture +Client - Reactors in Detail](../../under-the-hood/contexture-client.md#reactors-in-detail). + +[↪ Next: Types and Type Components](../../types/README.md) diff --git a/types/README.md b/types/README.md index b4b97ee..ede9f49 100644 --- a/types/README.md +++ b/types/README.md @@ -4,7 +4,8 @@ Table of Contents: - [DIY Types](diy-types.md) - - [How to Write a Type](diy-types.md#how-to-wite-a-type) + - [How to Write a Provider Type](diy-types.md#how-to-wite-a-provider-type) + - [How to Write a Client Type](diy-types.md#how-to-wite-a-client-type) - [How to Write a UI Component for a Type](diy-types.md#how-to-write-a-ui-component-for-a-type) - [The Example Types](diy-types.md#the-example-types) - [ElasticSearch Example Types](elasticsearch-example-types.md) diff --git a/types/diy-types.md b/types/diy-types.md index 267b7a2..2c4094c 100644 --- a/types/diy-types.md +++ b/types/diy-types.md @@ -16,13 +16,11 @@ We believe that making a generic framework will allow users to be creative on their search solutions. Because of that, we will start this document by explaining how to build your own types. -## How to Wite a Type - Writing a new single type is about writing two plain JavaScript Objects: - One which is sent to the [Contexture Provider](../querying/available-providers.md). - Another one which is sent to the initialization of the [Context Tree](../interactive-queries/contexture-client.md#context-tree). -### Provider Type +## How to Wite a Provider Type Initializing [Contexture Core](../querying/contexture-core.md) requires you to send a bunch of types per provider. A type on any @@ -66,7 +64,7 @@ and re-use code even if we switch the target database of the search. Once you have a type defined for one or more providers, you should write the same type for `contexture-client`. -### Contexture Client Type +## How to Write a Client Type Contexture Client already provides a some types based on our `Example Types` (more on that later.) These type definitions help @@ -82,6 +80,10 @@ might need for each one of the following properties: | `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | | `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | +More details about `contexture-client` types, their properties and +their reserved words can be seen on the README of +[contexture-client](https://github.com/smartprocure/contexture-client#client-types). + The example types are already included in any instantiation of Contexture Client's Contexture Tree. However, you can add any type you need simply by extending the exposed `exampleTypes` with your own. diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index 469f1df..48656e1 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -16,17 +16,299 @@ fulfill you expectations. Our ElasticSearch types are the following ones: -## [bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js) +## Bool Type The bool type is intended to work as an ElasticSearch terms aggregation with only one value for a single property. This is useful for user interfaces with a checkbox to include or exclude a specific field from a search (or a specific field-value pair). -Here's the definition of this type: +Here is the list of all properties this type uses: | Property Name | Type | Description | | --- | --- | --- | -| field | String | Name of the field that we will include or exclude from the search. | +| field | String | Name of the field that we will be using to filter the search search. | +| value | String | Value of the field that will be used to filter the search. | + +Example input: + +```javascript +{ + type: 'bool', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + term: { + fieldName: true + } +} +``` + +You can read more about it in: + +- [Source code of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js). +- [Unit tests of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/bool.js). +- Elastic Search [Term Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html). + +## Cardinality Type + +The Cardinality type serves to calculate an approximate count of the +distinct available values. This type only uses one property, `field`. + +Example input: +```javascript +{ + key: 'fieldName', + type: 'cardinality', + field: 'Organization.Name.untouched' +} +``` + +Example output: +```javascript +{ + aggs: { + cardinality: { + cardinality: { + field: 'Organization.Name.untouched' + } + } + } +} +``` + +- [Source code of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/cardinality.js). +- [Unit tests of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/cardinality.js). +- Elastic Search [Cardinality Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html). + +## Date Type + +The Date type is used to specify a range of dates that will be used to +filter the available results. This range of dates can be specified by +a string formatted date (`YYYY-MM-DD`) or by a small set of possible +humanly readable date ranges. Details follow: + +| Property Name | Type | Description | +| --- | --- | --- | +| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | +| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | +| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | +| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | + +Example input 1: +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25' +} +``` + +Example output 1: +```javascript +{ + range: { + fieldName: { + gte: '2016-04-25', + format: 'dateOptionalTime' + } + } +} +``` + +Example input 2: +```javascript +{ + type: 'date', + field: 'test', + from: 'thisQuarter', + useDateMath: true, +} +``` + +Example output 2: +```javascript +{ + range: { + test: { + gte: moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD'), + lte: moment.utc(datemath.parse(`${moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD')}||+3M-1d/d`)).format('YYYY-MM-DD'), + format: 'dateOptionalTime' + } + } +} +``` + +- [Source code of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). +- [Unit tests of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +- Elastic Search [Range QUery](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html). + +## Date Histogram Type + +The `dateHistogram` type + +| Property Name | Type | Description | +| --- | --- | --- | +| `key_field` | | | +| `value_field` | | | +| `interval` | | | +| `boundsRange_min` | | | +| `boundsRange_max` | | | +| `boundsRange_useDateMath` | | | + +Example input: +```javascript +{ + key: 'test', + type: 'dateHistogram', + key_field: 'PO.IssuedDate', + value_field: 'LineItem.TotalPrice' +} +``` + +Example output: +```javascript +{ + aggs: { + max_date: { + max: { + field: 'PO.IssuedDate', + }, + }, + min_date: { + min: { + field: 'PO.IssuedDate', + }, + }, + twoLevelAgg: { + date_histogram: { + field: 'PO.IssuedDate', + interval: 'year', + min_doc_count: 0, + }, + aggs: { + twoLevelAgg: { + stats: { + field: 'LineItem.TotalPrice', + }, + }, + }, + }, + }, +} +``` + +- [Source code of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). +- [Unit tests of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). +- [Two Level Aggregations in ElasticSearch](https://www.elastic.co/blog/intro-to-aggregations-pt-2-sub-aggregations). + +## Exists Type + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | | | +| `value` | | | + +Example input: +```javascript +{ + type: 'bool', + field: 'test', + value: true +} +``` + +Example output: +```javascript +{ + exists: { + field: 'test' + } +} +``` + +- [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). +- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). +- Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). + + +## Facet Type + +| Property Name | Type | Description | +| --- | --- | --- | +| `values` | | | +| `field` | | | +| `fieldMode` | | | +| `mode` | | | +| `size` | | | +| `sort` | | | +| `includeZeroes` | | | +| `optionsFilter` | | | +| `caseSensitive` | | | +| `anyOrder` | | | +| `maxWords` | | | +| `cardinality` | | | + +Example input 1: +```javascript +{ + key: 'test', + type: 'facet', + field: 'testField', + values: ['abc', '123'] +} +``` + +Example output 1: +```javascript +{ + terms: { + 'testField.untouched': ['abc', '123'] + } +} +``` + +Example input with exclude: +```javascript +{ + key: 'test', + type: 'facet', + field: 'testField', + mode: 'exclude', + values: ['abc', '123'], +} +``` + +Example output with exclude: +```javascript +{ + bool: { + must_not: { + terms: { + 'testField.untouched': ['abc', '123'], + }, + }, + }, +} +``` + +TODO: +- Explain missing values. +- Explain find filter box. + +- [Source code of the type: + facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +- [Unit tests of the type: + facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). + +TODO: +- All the other types! [↪ Next: Mongo Example Types](mongo-example-types.md) diff --git a/under-the-hood/README.md b/under-the-hood/README.md index ddcc8dc..5213f1b 100644 --- a/under-the-hood/README.md +++ b/under-the-hood/README.md @@ -13,5 +13,9 @@ Table of Contents: - [Contexture Mongo](contexture-providers/contexture-mongo.md) - [Building Your Own Provider](contexture-providers/building-your-own-provider.md) - [Contexture Client](contexture-client.md) - - [Reactors in detail](contexture-client.md#reactors-in-detail) + - [State Properties](contexture-client.md#state-properties) + - [Lenses](contexture-client.md#lenses) + - [Reactors in Detail](contexture-client.md#reactors-in-detail) + - [Serialization](contexture-client.md#serialization) + - [Traversals](contexture-client.md#traversals) - [Contexture React](contexture-react.md) From 27fee083445eb0b3fdf7aa0c77d108ea5481c836 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 1 Jun 2018 21:44:12 +0000 Subject: [PATCH 076/150] A little bit more on the exists type --- types/elasticsearch-example-types.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index 48656e1..92d500f 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -210,10 +210,13 @@ Example output: ## Exists Type +The `exists` type is used to check wether some property exsits or not. +It requires only two fields: `field` and `value`. + | Property Name | Type | Description | | --- | --- | --- | -| `field` | | | -| `value` | | | +| `field` | String | The target field we want to check. | +| `value` | Boolean | the value we want to check. Normally true or false. | Example input: ```javascript From a4ca7aae8d854f4b10abc614a253d9d068f659d1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Fri, 1 Jun 2018 22:50:25 +0000 Subject: [PATCH 077/150] Ground for future content --- types/elasticsearch-example-types.md | 59 ++++++++++++++++++++++++++++ types/mongo-example-types.md | 18 ++++++++- 2 files changed, 76 insertions(+), 1 deletion(-) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index 92d500f..ae3e64e 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -228,6 +228,7 @@ Example input: ``` Example output: + ```javascript { exists: { @@ -311,6 +312,64 @@ TODO: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). - Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). +## Geo Type + +| Property Name | Type | Description | +| --- | --- | --- | + +Example input: +```javascript +{ + type: 'geo', + field: 'test', + location: 'SmartProcure', + radius: 10, + operator: 'within', +} +``` + +Example output: + +```javascript +{ + geo_distance: { + test: '26.3170479,-80.1131784', + distance: '10mi', + }, +} +``` + +- [Source code of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/geo.js). +- [Unit tests of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/geo.js). + +## Number Type + +| Property Name | Type | Description | +| --- | --- | --- | + +Example input: +```javascript +{ + type: 'number', + field: 'test', + min: 500, + max: 1000, +} +``` + +Example output: + +```javascript +{ + range: { + test: { + gte: 500, + lte: 1000, + }, + }, +} +``` + TODO: - All the other types! diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md index 5ddd4cd..9acd952 100644 --- a/types/mongo-example-types.md +++ b/types/mongo-example-types.md @@ -1,6 +1,22 @@ [↩ Parent: Table of Contents](../README.md) [↩ Previous: ElasticSearch Example Types](elasticsearch-example-types.md) -# MongoDB Example Types +## Type +| Property Name | Type | Description | +| --- | --- | --- | + +Example input: +```javascript +``` + +Example output: + +```javascript +``` + +- [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). +- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). +- Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). + [↪ Next: Available React Components for Types](react-components.md) From fbe7bf19b4cf8c2cd1dc4c2b0993854615fdaf7c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 16:32:50 +0000 Subject: [PATCH 078/150] first commit of the week --- types/elasticsearch-example-types.md | 2 +- types/mongo-example-types.md | 2 +- types/react-components.md | 54 +++++++++++++++++++++++++++- 3 files changed, 55 insertions(+), 3 deletions(-) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index ae3e64e..14f440d 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -369,7 +369,7 @@ Example output: }, } ``` - + TODO: - All the other types! diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md index 9acd952..a7055ab 100644 --- a/types/mongo-example-types.md +++ b/types/mongo-example-types.md @@ -18,5 +18,5 @@ Example output: - [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). - [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). - Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). - + [↪ Next: Available React Components for Types](react-components.md) diff --git a/types/react-components.md b/types/react-components.md index 2826873..7b11b19 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -1,6 +1,58 @@ [↩ Parent: Table of Contents](../README.md) [↩ Previous: Mongo Example Types](mongo-example-types.md) -# MongoDB Example Types +# Available React Components for Types + +A huge part of working with advanced search interfaces is identifying +how to portray a specific search filter that you might want to use. +We've talked about very simple components that react to changes on the +tree state, but now we'll see how to make it easier for more complex +nodes to gather the inputs necessary, and also to show the results +correctly. Here we'll see components specifically crafted for some of +our providers' example types. + +## Date + +![Date Type Screenshot 1](https://i.imgur.com/XwuGi2c.png) +![Date Type Screenshot 2](https://i.imgur.com/joTECy0.png) +![Date Type Screenshot 3](https://i.imgur.com/acvIuIS.png) + +(Purpose) + +Here's how you write a node of type `date` in your _searchTree_: +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25', + to: '2017-05-26' +} +``` + +Here is the list of properties that it expects to have on the node: + +| Property Name | Type | Description | +| --- | --- | --- | + +Here's how you write your component: +```javascript +let DateComponent = require('contexture-react/dist/exampleTypes').Date + +// Then, on your render function, or where you put your components: + +``` + +- [Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). +- [Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +- [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). + +## DateHistogram +## Facet +## Number +## Query +## ResultCount +## ResultPager +## ResultTable +## TermsStats [↪ Next: Other Components](../other-components/README.md) From 8234bc4384bfe0f30a2fd311bf477f7bfdbb1cfc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 16:34:34 +0000 Subject: [PATCH 079/150] lil changes --- types/react-components.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/types/react-components.md b/types/react-components.md index 7b11b19..8e1bcc6 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -15,7 +15,9 @@ our providers' example types. ![Date Type Screenshot 1](https://i.imgur.com/XwuGi2c.png) ![Date Type Screenshot 2](https://i.imgur.com/joTECy0.png) -![Date Type Screenshot 3](https://i.imgur.com/acvIuIS.png) + +_(Same goes with the right, not adding another screenshot to avoid +consuming more space.)_ (Purpose) From 59dc3cf97634a507cf12a6e96a221764265b7b1e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 17:25:50 +0000 Subject: [PATCH 080/150] DateHistogram --- types/react-components.md | 93 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 90 insertions(+), 3 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index 8e1bcc6..4e9a19d 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -19,7 +19,7 @@ our providers' example types. _(Same goes with the right, not adding another screenshot to avoid consuming more space.)_ -(Purpose) +(TODO: Purpose) Here's how you write a node of type `date` in your _searchTree_: ```javascript @@ -31,15 +31,21 @@ Here's how you write a node of type `date` in your _searchTree_: } ``` -Here is the list of properties that it expects to have on the node: +Here is the list of properties that this component expects to have on the node: | Property Name | Type | Description | | --- | --- | --- | +| `from` | Date or String (`YYYY-MM-DD` format) | The initial date of our timeframe. | +| `to` | Date or String (`YYYY-MM-DD` format) | The final date of our timeframe. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the provider's type. See the Provider type's +documentation in our [previous pages](README.md). Here's how you write your component: ```javascript let DateComponent = require('contexture-react/dist/exampleTypes').Date - +// ... // Then, on your render function, or where you put your components: ``` @@ -49,6 +55,87 @@ let DateComponent = require('contexture-react/dist/exampleTypes').Date - [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). ## DateHistogram + +![Date Histogram Screenshot](https://i.imgur.com/XwuGi2c.png) + +(TODO: Purpose) + +Here's how you write a node of type `date` in your _searchTree_: +```javascript +{ + key: 'releases', + type: 'dateHistogram', + key_field: 'released', + value_field: 'imdbVotes', + interval: '3650d', + context: { + entries: [ + { + key: 0, + doc_count: 1, + count: 1, + min: 625633, + max: 625633, + avg: 625633, + sum: 625633, + }, + { + key: 315360000000, + doc_count: 3, + count: 3, + min: 74450, + max: 557731, + avg: 355868.3333333333, + sum: 1067605, + }, + { + key: 630720000000, + doc_count: 2, + count: 2, + min: 82360, + max: 376362, + avg: 229361, + sum: 458722, + }, + { + key: 946080000000, + doc_count: 4, + count: 4, + min: 28087, + max: 395463, + avg: 275019.25, + sum: 1100077, + }, + { + key: 1261440000000, + doc_count: 1, + count: 1, + min: 264551, + max: 264551, + avg: 264551, + sum: 264551, + }, + ], + maxDate: null, + minDate: null, + }, +} +``` + +Since this component doesn't need any property from the node itself, +but only from the results, here's how you write your component: +```javascript +let DateHistogram = require('contexture-react/dist/exampleTypes').DateHistogram +let formatYear = x => new Date(x).getFullYear() + 1 +// ... +// Then, on your render function, or where you put your components: + +``` + +- [Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). +- [Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). +- [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). + ## Facet ## Number ## Query From f7f07ef0c5b8995c5ec91e19ce5345b87a9dde95 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 17:45:39 +0000 Subject: [PATCH 081/150] Facet --- types/react-components.md | 106 ++++++++++++++++++-------------------- 1 file changed, 50 insertions(+), 56 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index 4e9a19d..902d65d 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -11,6 +11,10 @@ nodes to gather the inputs necessary, and also to show the results correctly. Here we'll see components specifically crafted for some of our providers' example types. +**Note:** Keep in mind that the styles of these components is purely +optional. Later on we'll see how to [style our +components](../theming/README.md). + ## Date ![Date Type Screenshot 1](https://i.imgur.com/XwuGi2c.png) @@ -33,13 +37,13 @@ Here's how you write a node of type `date` in your _searchTree_: Here is the list of properties that this component expects to have on the node: -| Property Name | Type | Description | +| Property Name | Type | Required | Description | | --- | --- | --- | -| `from` | Date or String (`YYYY-MM-DD` format) | The initial date of our timeframe. | -| `to` | Date or String (`YYYY-MM-DD` format) | The final date of our timeframe. | +| `from` | Date or String (`YYYY-MM-DD` format) | Yes | The initial date of our timeframe. | +| `to` | Date or String (`YYYY-MM-DD` format) | No | The final date of our timeframe. | **Note:** The properties present in the search tree that aren't used by the node -might be needed for the provider's type. See the Provider type's +might be needed for the Provider's type. See the Provider type's documentation in our [previous pages](README.md). Here's how you write your component: @@ -60,7 +64,7 @@ let DateComponent = require('contexture-react/dist/exampleTypes').Date (TODO: Purpose) -Here's how you write a node of type `date` in your _searchTree_: +Here's how you write a node of type `dateHistogram` in your _searchTree_: ```javascript { key: 'releases', @@ -68,60 +72,13 @@ Here's how you write a node of type `date` in your _searchTree_: key_field: 'released', value_field: 'imdbVotes', interval: '3650d', - context: { - entries: [ - { - key: 0, - doc_count: 1, - count: 1, - min: 625633, - max: 625633, - avg: 625633, - sum: 625633, - }, - { - key: 315360000000, - doc_count: 3, - count: 3, - min: 74450, - max: 557731, - avg: 355868.3333333333, - sum: 1067605, - }, - { - key: 630720000000, - doc_count: 2, - count: 2, - min: 82360, - max: 376362, - avg: 229361, - sum: 458722, - }, - { - key: 946080000000, - doc_count: 4, - count: 4, - min: 28087, - max: 395463, - avg: 275019.25, - sum: 1100077, - }, - { - key: 1261440000000, - doc_count: 1, - count: 1, - min: 264551, - max: 264551, - avg: 264551, - sum: 264551, - }, - ], - maxDate: null, - minDate: null, - }, } ``` +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. See the Provider type's +documentation in our [previous pages](README.md). + Since this component doesn't need any property from the node itself, but only from the results, here's how you write your component: ```javascript @@ -137,6 +94,43 @@ let formatYear = x => new Date(x).getFullYear() + 1 - [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). ## Facet + +![Facet Type Screenshot](https://i.imgur.com/1X3mrfq.png) + +(TODO: Purpose) + +Here's how you write a node of type `facet` in your _searchTree_: +```javascript +{ + key: 'facet', + type: 'facet', + values: ['a'], +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | +| `values` | Array | Yes | Array of selected values. To have a value selected by default, you will need to know in advance which value to put here, otherwise you can leave this with an empty array and let users select the values themselves. | +| `size` | Number | No | Max number of options to display. The default is 10. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. See the Provider type's +documentation in our [previous pages](README.md). + +Here's how you write your component: +```javascript +let Facet = require('contexture-react/dist/exampleTypes').Facet +// ... +// Then, on your render function, or where you put your components: + +``` + +- [Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +- [Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- [Source code of the Facet component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Facet.js). + ## Number ## Query ## ResultCount From 9fdf2277ada71f1b2f662e4fdc0d0c8c3e7c9f8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 17:49:39 +0000 Subject: [PATCH 082/150] Fixing the table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 💁‍♀️is this woodworking? 🦋 --- types/react-components.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index 902d65d..7c3e41b 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -38,7 +38,7 @@ Here's how you write a node of type `date` in your _searchTree_: Here is the list of properties that this component expects to have on the node: | Property Name | Type | Required | Description | -| --- | --- | --- | +| --- | --- | --- | --- | | `from` | Date or String (`YYYY-MM-DD` format) | Yes | The initial date of our timeframe. | | `to` | Date or String (`YYYY-MM-DD` format) | No | The final date of our timeframe. | @@ -111,7 +111,7 @@ Here's how you write a node of type `facet` in your _searchTree_: Here is the list of properties that this component expects to have on the node: | Property Name | Type | Required | Description | -| --- | --- | --- | +| --- | --- | --- | --- | | `values` | Array | Yes | Array of selected values. To have a value selected by default, you will need to know in advance which value to put here, otherwise you can leave this with an empty array and let users select the values themselves. | | `size` | Number | No | Max number of options to display. The default is 10. | From b5330a50cfe6d8c844bde8f99d1831c94c734b23 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 17:52:35 +0000 Subject: [PATCH 083/150] Tweak --- types/react-components.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/types/react-components.md b/types/react-components.md index 7c3e41b..1bf7d2b 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -58,7 +58,7 @@ let DateComponent = require('contexture-react/dist/exampleTypes').Date - [Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). - [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). -## DateHistogram +## Date Histogram ![Date Histogram Screenshot](https://i.imgur.com/XwuGi2c.png) From f527f15fd55b7f20a83ca0299aa941909e9ae652 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 17:53:35 +0000 Subject: [PATCH 084/150] Corrected Date Histogram's image --- types/react-components.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/types/react-components.md b/types/react-components.md index 1bf7d2b..32eec40 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -60,7 +60,7 @@ let DateComponent = require('contexture-react/dist/exampleTypes').Date ## Date Histogram -![Date Histogram Screenshot](https://i.imgur.com/XwuGi2c.png) +![Date Histogram Screenshot](https://i.imgur.com/oZsXY5R.png) (TODO: Purpose) From a8a2d44204dd0d3d5e6150889b4823476b27a414 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 18:22:50 +0000 Subject: [PATCH 085/150] Number type --- types/react-components.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/types/react-components.md b/types/react-components.md index 32eec40..87c0269 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -132,6 +132,45 @@ let Facet = require('contexture-react/dist/exampleTypes').Facet - [Source code of the Facet component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Facet.js). ## Number + +![Number Type Screenshot](https://i.imgur.com/Uuu16wy.png) + +(TODO: Purpose) + +Here's how you write a node of type `number` in your _searchTree_: +```javascript +{ + key: 'searchNumber', + type: 'number', + field: 'metaScore', + min: 0, + max: 100, +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. See the Provider type's +documentation in our [previous pages](README.md). + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/exampleTypes').Number +// ... +// Then, on your render function, or where you put your components: + +``` + +- [Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). +- [Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). +- [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). + ## Query ## ResultCount ## ResultPager From 7890a3757f5c739a68a5a1f8b7814a8d59e6b455 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 19:09:32 +0000 Subject: [PATCH 086/150] Query type --- types/react-components.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/types/react-components.md b/types/react-components.md index 87c0269..f10d0ff 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -172,6 +172,43 @@ let Number = require('contexture-react/dist/exampleTypes').Number - [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). ## Query + +![Query Type Screenshot](https://i.imgur.com/8r2X9MI.png) + +(TODO: Purpose) + +Here's how you write a node of type `query` in your _searchTree_: +```javascript +{ + key: 'searchQuery', + type: 'query', + field: 'title', + query: '' +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `query` | String | No | Search query that should be visible in the component. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. See the Provider type's +documentation in our [previous pages](README.md). + +Here's how you write your component: +```javascript +let Query = require('contexture-react/dist/exampleTypes').Query +// ... +// Then, on your render function, or where you put your components: + +``` + +- [Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +- [Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). +- [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). + ## ResultCount ## ResultPager ## ResultTable From 544e3f0d444d4687acda55282b6c489a07771685 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 20:08:03 +0000 Subject: [PATCH 087/150] Almost done with the react components --- types/react-components.md | 123 ++++++++++++++++++++++++++++++++------ 1 file changed, 105 insertions(+), 18 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index f10d0ff..617c677 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -50,17 +50,19 @@ Here's how you write your component: ```javascript let DateComponent = require('contexture-react/dist/exampleTypes').Date // ... -// Then, on your render function, or where you put your components: +// Later, on your render function, or where you put your components: ``` -- [Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). -- [Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +- [(ElasticSearch Provider) Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). +- [(ElasticSearch Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +- [(MongoDb Provider) Source code of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). +- [(MongoDb Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). - [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). -## Date Histogram +## DateHistogram -![Date Histogram Screenshot](https://i.imgur.com/oZsXY5R.png) +![DateHistogram Screenshot](https://i.imgur.com/oZsXY5R.png) (TODO: Purpose) @@ -85,12 +87,12 @@ but only from the results, here's how you write your component: let DateHistogram = require('contexture-react/dist/exampleTypes').DateHistogram let formatYear = x => new Date(x).getFullYear() + 1 // ... -// Then, on your render function, or where you put your components: +// Later, on your render function, or where you put your components: ``` -- [Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). -- [Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). +- [(ElasticSearch Provider) Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). +- [(ElasticSearch Provider) Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). - [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). ## Facet @@ -123,12 +125,14 @@ Here's how you write your component: ```javascript let Facet = require('contexture-react/dist/exampleTypes').Facet // ... -// Then, on your render function, or where you put your components: +// Later, on your render function, or where you put your components: ``` -- [Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). -- [Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- [(ElasticSearch Provider) Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +- [(ElasticSearch Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- [(MongoDb Provider) Source code of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). +- [(MongoDb Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/facet.js). - [Source code of the Facet component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Facet.js). ## Number @@ -163,12 +167,14 @@ Here's how you write your component: ```javascript let Number = require('contexture-react/dist/exampleTypes').Number // ... -// Then, on your render function, or where you put your components: +// Later, on your render function, or where you put your components: ``` -- [Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). -- [Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). +- [(ElasticSearch Provider) Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). +- [(ElasticSearch Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). +- [(MongoDb Provider) Source code of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). +- [(MongoDb Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). - [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). ## Query @@ -201,17 +207,98 @@ Here's how you write your component: ```javascript let Query = require('contexture-react/dist/exampleTypes').Query // ... -// Then, on your render function, or where you put your components: +// Later, on your render function, or where you put your components: ``` -- [Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). -- [Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). +- [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +- [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). +- [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). +- [(MongoDb Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/query.js). - [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). ## ResultCount + +![ResultCount Type Screenshot](https://i.imgur.com/htuXprD.png) + +(TODO: Purpose) + +Most of your Contexture Trees will have a node with type `results`. +This node posesses information such as the resulting records +themselves, but also which page you're in, how many +elements are per page you will receive, and the total records there +are for this given query, which is what we're looking for for this +type. + +All of these properties are automatically writen by the Contexture +architecture, so we'll be simply passing the tree and the path to the +results type node: + +```javascript +let ResultCount = require('contexture-react/dist/exampleTypes').ResultCount +// ... +// Later, on your render function, or where you put your components: + +``` + +- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). +- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). +- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). +- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). +- [Source code of the ResultCount component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultCount.js). + ## ResultPager + +![ResultPager Type Screenshot](https://i.imgur.com/h73I2QZ.png) + +(TODO: Purpose) + +Most of your Contexture Trees will have a node with type `results`. +This node posesses information such as the resulting records +themselves, but also which page you're in, how many +elements are per page you will receive, and the total records there +are for this given query. We use a combination of those values to get +how many pages we can move forward (and backwards), and also to move +around these pages (since our trees react in real time). + +All of these properties are automatically writen by the Contexture +architecture, so we'll be simply passing the tree and the path to the +results type node: + +```javascript +let ResultPager = require('contexture-react/dist/exampleTypes').ResultPager +// ... +// Later, on your render function, or where you put your components: + +``` + +- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). +- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). +- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). +- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). +- [Source code of the ResultPager component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultPager.js). + ## ResultTable -## TermsStats + +![resulttable type screenshot](https://i.imgur.com/cc5urub.png) + +(todo: purpose) + +the results are automatically writen by the contexture architecture, +so we'll be simply passing the tree and the path to the results type +node: + +```javascript +let resulttable = require('contexture-react/dist/exampletypes').resulttable +// ... +// later, on your render function, or where you put your components: + +``` + +- [(ElasticSearch Provider) source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). +- [(elasticSearch Provider) unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). +- [(MongoDb Provider) source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). +- [(MongoDb Provider) unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). +- [Source code of the ResultTable component](https://github.com/smartprocure/contexture-react/blob/master/src/exampletypes/ResultTable.js). [↪ Next: Other Components](../other-components/README.md) From cd662ca792e2bf5e4b606ccc9e9ca506d3367f4d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 20:18:23 +0000 Subject: [PATCH 088/150] Let's see how it looks --- types/react-components.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index 617c677..05c6aae 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -11,8 +11,8 @@ nodes to gather the inputs necessary, and also to show the results correctly. Here we'll see components specifically crafted for some of our providers' example types. -**Note:** Keep in mind that the styles of these components is purely -optional. Later on we'll see how to [style our +**Note:** Keep in mind that the theme on these components is purely +optional. Later on we'll see how to [theme our components](../theming/README.md). ## Date @@ -23,9 +23,10 @@ components](../theming/README.md). _(Same goes with the right, not adding another screenshot to avoid consuming more space.)_ -(TODO: Purpose) - -Here's how you write a node of type `date` in your _searchTree_: +The Date component helps by allowing users to filter the data to +obtain results within a specific range of dates. It consists of only +two date pickers. Here's how you write a node of type `date` in your +_searchTree_: ```javascript { type: 'date', From d02fde50517215b9ce0b375eaa1a8c7a726c9049 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 20:41:53 +0000 Subject: [PATCH 089/150] The descriptions --- types/react-components.md | 114 +++++++++++++++++++++++--------------- 1 file changed, 68 insertions(+), 46 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index 05c6aae..1dfd61b 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -15,6 +15,49 @@ our providers' example types. optional. Later on we'll see how to [theme our components](../theming/README.md). +## Query + +![Query Type Screenshot](https://i.imgur.com/8r2X9MI.png) + +The Query component is probably the most commonly needed for search +interfaces. It's an input field that allows you to filter results if +the text matches part of the value of a specfic property on any of the +records. + +Here's how you write a node of type `query` in your _searchTree_: +```javascript +{ + key: 'searchQuery', + type: 'query', + field: 'title', + query: '' +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `query` | String | No | Search query that should be visible in the component. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. See the Provider type's +documentation in our [previous pages](README.md). + +Here's how you write your component: +```javascript +let Query = require('contexture-react/dist/exampleTypes').Query +// ... +// Later, on your render function, or where you put your components: + +``` + +- [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +- [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). +- [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). +- [(MongoDb Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/query.js). +- [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). + ## Date ![Date Type Screenshot 1](https://i.imgur.com/XwuGi2c.png) @@ -65,7 +108,11 @@ let DateComponent = require('contexture-react/dist/exampleTypes').Date ![DateHistogram Screenshot](https://i.imgur.com/oZsXY5R.png) -(TODO: Purpose) +The DateHistogram component is about representing how many records +were found during what periods of time. This component currently +doesn't offer interactive features, but you could use it as +inspiration to write another one that would allow you to dive in these +date ranges to see what records appear for each one of them. Here's how you write a node of type `dateHistogram` in your _searchTree_: ```javascript @@ -100,7 +147,9 @@ let formatYear = x => new Date(x).getFullYear() + 1 ![Facet Type Screenshot](https://i.imgur.com/1X3mrfq.png) -(TODO: Purpose) +The Facet component allows users to filter the results by picking a +specific common option among all the values that the records might +have for a specific field. Here's how you write a node of type `facet` in your _searchTree_: ```javascript @@ -140,7 +189,9 @@ let Facet = require('contexture-react/dist/exampleTypes').Facet ![Number Type Screenshot](https://i.imgur.com/Uuu16wy.png) -(TODO: Purpose) +The Number component allows users to specify a numeric range to filter +the data based on the available results which values fit within this +range for a specific field. Here's how you write a node of type `number` in your _searchTree_: ```javascript @@ -178,51 +229,13 @@ let Number = require('contexture-react/dist/exampleTypes').Number - [(MongoDb Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). - [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). -## Query - -![Query Type Screenshot](https://i.imgur.com/8r2X9MI.png) - -(TODO: Purpose) - -Here's how you write a node of type `query` in your _searchTree_: -```javascript -{ - key: 'searchQuery', - type: 'query', - field: 'title', - query: '' -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `query` | String | No | Search query that should be visible in the component. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. See the Provider type's -documentation in our [previous pages](README.md). - -Here's how you write your component: -```javascript -let Query = require('contexture-react/dist/exampleTypes').Query -// ... -// Later, on your render function, or where you put your components: - -``` - -- [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). -- [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). -- [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). -- [(MongoDb Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/query.js). -- [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). - ## ResultCount ![ResultCount Type Screenshot](https://i.imgur.com/htuXprD.png) -(TODO: Purpose) +The ResultCount component will only show you the number of visible +results compared to the number of total results. It's not an +interactive component. Most of your Contexture Trees will have a node with type `results`. This node posesses information such as the resulting records @@ -252,7 +265,14 @@ let ResultCount = require('contexture-react/dist/exampleTypes').ResultCount ![ResultPager Type Screenshot](https://i.imgur.com/h73I2QZ.png) -(TODO: Purpose) +The ResultPager component is an interactive component that will show +you which page you're at (in the middle) and what pages are around +your current page, as well as some controllers to move forward and +backwards. + +The style of this component isn't very friendly, but it's very easy to +customize. For more about style customization, please visit our +[theming docs](../theming/README.md). Most of your Contexture Trees will have a node with type `results`. This node posesses information such as the resulting records @@ -283,7 +303,9 @@ let ResultPager = require('contexture-react/dist/exampleTypes').ResultPager ![resulttable type screenshot](https://i.imgur.com/cc5urub.png) -(todo: purpose) +The ResultTable is a component that will display a table with all the +available results, in which each of the result values will be +displayed as columns. the results are automatically writen by the contexture architecture, so we'll be simply passing the tree and the path to the results type From 811c38fa44335af67de894ed05ca2b5e90e895f3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 20:42:24 +0000 Subject: [PATCH 090/150] Some TODOs --- types/react-components.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/types/react-components.md b/types/react-components.md index 1dfd61b..a3055cf 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -274,6 +274,8 @@ The style of this component isn't very friendly, but it's very easy to customize. For more about style customization, please visit our [theming docs](../theming/README.md). +(TODO: Customization options) + Most of your Contexture Trees will have a node with type `results`. This node posesses information such as the resulting records themselves, but also which page you're in, how many @@ -307,6 +309,8 @@ The ResultTable is a component that will display a table with all the available results, in which each of the result values will be displayed as columns. +(TODO: Customization options) + the results are automatically writen by the contexture architecture, so we'll be simply passing the tree and the path to the results type node: From 9a945ab9520dbf20ccaaacdb3d0d19bbacbd6289 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Mon, 4 Jun 2018 21:47:50 +0000 Subject: [PATCH 091/150] Customization properties and comment about Components --- types/react-components.md | 82 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 76 insertions(+), 6 deletions(-) diff --git a/types/react-components.md b/types/react-components.md index a3055cf..5326ac3 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -11,9 +11,14 @@ nodes to gather the inputs necessary, and also to show the results correctly. Here we'll see components specifically crafted for some of our providers' example types. -**Note:** Keep in mind that the theme on these components is purely +**Notes:** +- Keep in mind that the theme on these components is purely optional. Later on we'll see how to [theme our components](../theming/README.md). +- Please be aware that when we refer to `Component`, we mean a + Function that returns a valid JSX Element. You can read more here: + [Components and + Props, on the ReactJS docs](https://reactjs.org/docs/components-and-props.html). ## Query @@ -52,6 +57,15 @@ let Query = require('contexture-react/dist/exampleTypes').Query ``` +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `TextInput` | Component | `input` | Text input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | + +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). - [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). - [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). @@ -98,6 +112,15 @@ let DateComponent = require('contexture-react/dist/exampleTypes').Date ``` +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `DateInput` | Component | `x => ` | The component that wraps each one of the inputs where the dates end up written by the date-picker. | + +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). - [(ElasticSearch Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). - [(MongoDb Provider) Source code of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). @@ -139,6 +162,19 @@ let formatYear = x => new Date(x).getFullYear() + 1 ``` +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `background` | Function | `(record, max) => '#ccc'` | A function that returns the background color that is used to render each one of the bars in the resulting chart. | +| `height` | Number | `100` | Specifies the max height of the whole chart. | +| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | +| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | +| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | + +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). - [(ElasticSearch Provider) Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). - [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). @@ -179,6 +215,13 @@ let Facet = require('contexture-react/dist/exampleTypes').Facet ``` +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `hide.facetFilter` | Object (with an optional single property, `facetFilter`) | `{}` | Allows you to hide the text input that helps on searching for the available options. This text input is very valuable when the results are larger than the available visible results. | +| `TextInput` | Component | `input` | Allows you to customize the text input. | + +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). - [(ElasticSearch Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). - [(MongoDb Provider) Source code of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). @@ -223,6 +266,15 @@ let Number = require('contexture-react/dist/exampleTypes').Number ``` +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `NumberInput` | Component | `x => ` | Number input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | + +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). - [(ElasticSearch Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). - [(MongoDb Provider) Source code of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). @@ -255,6 +307,8 @@ let ResultCount = require('contexture-react/dist/exampleTypes').ResultCount ``` +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). - [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). - [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). @@ -271,11 +325,9 @@ your current page, as well as some controllers to move forward and backwards. The style of this component isn't very friendly, but it's very easy to -customize. For more about style customization, please visit our +customize. For more about theme changes, please visit our [theming docs](../theming/README.md). -(TODO: Customization options) - Most of your Contexture Trees will have a node with type `results`. This node posesses information such as the resulting records themselves, but also which page you're in, how many @@ -295,6 +347,17 @@ let ResultPager = require('contexture-react/dist/exampleTypes').ResultPager ``` +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `List` | Component | `div` | The component that wraps the whole list of pages. | +| `Item` | Component | `span` | The component that wraps each of the available page numbers. | +| `Link` | Component | `a` | An element that wraps each one of the pagination controls. | + +To read more, check the following links: + - [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). - [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). - [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). @@ -309,8 +372,6 @@ The ResultTable is a component that will display a table with all the available results, in which each of the result values will be displayed as columns. -(TODO: Customization options) - the results are automatically writen by the contexture architecture, so we'll be simply passing the tree and the path to the results type node: @@ -322,6 +383,15 @@ let resulttable = require('contexture-react/dist/exampletypes').resulttable ``` +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `Table` | Component | `table` | The component that wraps the table list of results. | + +To read more, check the following links: + - [(ElasticSearch Provider) source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). - [(elasticSearch Provider) unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). - [(MongoDb Provider) source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). From 81690116f09595b45eb626b55f24252a4922dd6c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 5 Jun 2018 17:56:26 +0000 Subject: [PATCH 092/150] More info on the ES types --- types/elasticsearch-example-types.md | 238 +++++++++++++++++++++------ 1 file changed, 191 insertions(+), 47 deletions(-) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index 14f440d..e619f7f 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -60,11 +60,12 @@ You can read more about it in: The Cardinality type serves to calculate an approximate count of the distinct available values. This type only uses one property, `field`. +It returns it's values in the `context` of the node, rather than in +the `results` node. Example input: ```javascript { - key: 'fieldName', type: 'cardinality', field: 'Organization.Name.untouched' } @@ -83,6 +84,8 @@ Example output: } ``` +You can read more about it in: + - [Source code of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/cardinality.js). - [Unit tests of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/cardinality.js). - Elastic Search [Cardinality Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html). @@ -126,7 +129,7 @@ Example input 2: ```javascript { type: 'date', - field: 'test', + field: 'fieldName', from: 'thisQuarter', useDateMath: true, } @@ -136,7 +139,7 @@ Example output 2: ```javascript { range: { - test: { + fieldName: { gte: moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD'), lte: moment.utc(datemath.parse(`${moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD')}||+3M-1d/d`)).format('YYYY-MM-DD'), format: 'dateOptionalTime' @@ -145,27 +148,37 @@ Example output 2: } ``` +You can read more about it in: + - [Source code of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). - [Unit tests of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). - Elastic Search [Range QUery](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html). ## Date Histogram Type -The `dateHistogram` type +The `dateHistogram` type is very useful for retrieving the number of +results within specific periods of time. The idea here is to end up +building a chart showing how records have changed over time, for +example by it's creation date or some other date property. This type +is able to do this by running a nested stats aggregation inside a +dateHistogram aggregation, while also supporting for tweaking min/max +bounds. + +This type returns it's values in the `context` of the node, rather +than in the `results` node. | Property Name | Type | Description | | --- | --- | --- | -| `key_field` | | | -| `value_field` | | | -| `interval` | | | -| `boundsRange_min` | | | -| `boundsRange_max` | | | -| `boundsRange_useDateMath` | | | +| `key_field` | String | What might be considered a good candidate for the X axis of the chart. | +| `value_field` | String | What might be considered a good candidate for the Y axis of the chart. | +| `interval` | | String | Available expressions for interval: `year` (`1y`), `quarter` (`1q`), `month` (`1M`), `week` (`1w`), `day` (`1d`), `hour` (`1h`), `minute` (`1m`), `second` (`1s`). | +| `boundsRange_min` | Date or String | Lower date limit that will be considered to filter the possible results. | +| `boundsRange_max` | Date or String | Upper date limit that will be considered to filter the possible results. | +| `boundsRange_useDateMath` | Boolean | If set to `true`, the min and max ranges will be valid as strings representative of dates, and will be formatted by NPM's library [`@elastic/datemath`](https://github.com/elastic/datemath-js). | Example input: ```javascript { - key: 'test', type: 'dateHistogram', key_field: 'PO.IssuedDate', value_field: 'LineItem.TotalPrice' @@ -204,9 +217,12 @@ Example output: } ``` +You can read more about it in: + - [Source code of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). - [Unit tests of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). - [Two Level Aggregations in ElasticSearch](https://www.elastic.co/blog/intro-to-aggregations-pt-2-sub-aggregations). +- [Date Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html). ## Exists Type @@ -222,7 +238,7 @@ Example input: ```javascript { type: 'bool', - field: 'test', + field: 'fieldName', value: true } ``` @@ -232,39 +248,47 @@ Example output: ```javascript { exists: { - field: 'test' + field: 'fieldName' } } ``` +You can read more about it in: + - [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). - [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). - Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). - ## Facet Type -| Property Name | Type | Description | -| --- | --- | --- | -| `values` | | | -| `field` | | | -| `fieldMode` | | | -| `mode` | | | -| `size` | | | -| `sort` | | | -| `includeZeroes` | | | -| `optionsFilter` | | | -| `caseSensitive` | | | -| `anyOrder` | | | -| `maxWords` | | | -| `cardinality` | | | +The `facet` type represents a list of dynamic choices, e.g. a checkbox +list filter. We achieve this by running an ElasticSearch terms +aggregation. We provide a way to limit the number of results that you +will receive with the `size` property, so that large queries can +safely be used. For that same purpose, the property `optionsFilter` is +given, so that search queries can filter the results with a string. + +Facet returns it's values in the `context` of the node, rather than in +the `results` node. + +| Property | Type | Default | Description | +| ---- | ---- | ------- | ----------- | +| `field` | String | None, *required* | The field it's operating on. | +| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | +| `values` | Array (of strings) | `[]` | Already selected values. | +| `fieldMode` | String (`autocomplete`, `word` or `suggest`) | `autocomplete` | Whether to look at the entire field (`autocomplete`), the analyzed words in the field (`word`), or magic suggestions (`suggest`). | +| `size` | Number | 12 | How many options to return. | +| `cardinality` | Number | 5000 | Precision threshold override. | +| `includeZeroes` | Boolean | false | If true, it will include options with 0 matching documents (aka `min_doc_count: 0`) | +| `optionsFilter` | String | '' | Filters the options further, e.g. a find box above a checkbox list | +| `caseSensitive` | Boolean | false | Whether options filter is case sensitive. | +| `sort` | String (`term` or `count`) | `count` | Sort results alphabetically or by count of matching records. | Example input 1: ```javascript { - key: 'test', type: 'facet', - field: 'testField', + field: 'fieldName', values: ['abc', '123'] } ``` @@ -273,7 +297,7 @@ Example output 1: ```javascript { terms: { - 'testField.untouched': ['abc', '123'] + 'fieldName.untouched': ['abc', '123'] } } ``` @@ -281,9 +305,8 @@ Example output 1: Example input with exclude: ```javascript { - key: 'test', type: 'facet', - field: 'testField', + field: 'fieldName', mode: 'exclude', values: ['abc', '123'], } @@ -295,33 +318,37 @@ Example output with exclude: bool: { must_not: { terms: { - 'testField.untouched': ['abc', '123'], + 'fieldName.untouched': ['abc', '123'], }, }, }, } ``` -TODO: -- Explain missing values. -- Explain find filter box. +You can read more about it in: -- [Source code of the type: - facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). -- [Unit tests of the type: - facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- [Source code of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +- [Unit tests of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). - Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). ## Geo Type +The `geo` type represents a geographic radius search. It needs a +geocodeLocation service passed in to it. We currently assume that you +will be using a google maps geocoder search. + | Property Name | Type | Description | | --- | --- | --- | +| `field` | String (required) | The field it's operating on | +| `location` | String (required) | Location to geocode (e.g. an address, businessname, anything the google geocode can take) | +| `radius` | Number (required) | Radius in miles | +| `operator` | String (either `within` or `not within`) | Whether the filter forces inclusion or exclusion (defaults with `within`). | Example input: ```javascript { type: 'geo', - field: 'test', + field: 'fieldName', location: 'SmartProcure', radius: 10, operator: 'within', @@ -333,25 +360,41 @@ Example output: ```javascript { geo_distance: { - test: '26.3170479,-80.1131784', + fieldName: '26.3170479,-80.1131784', distance: '10mi', }, } ``` +You can read more about it in: + - [Source code of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/geo.js). - [Unit tests of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/geo.js). +- [ElasticSearch's Geo Distance Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-query.html). ## Number Type +Number represents a number range with inclusive bounds. This type +provides the ability to determine the best range values based on +percentile interval and range threshold. + | Property Name | Type | Description | | --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | + +Some Notes: +1. An empty value as the upper boundary represents infinity. +2. An empty value as the lower boundary represents negative infinity. +3. Zero has to be respected as a boundary value. +4. If findBestRange is true it will return the best min and max range. Example input: ```javascript { type: 'number', - field: 'test', + field: 'fieldName', min: 500, max: 1000, } @@ -362,7 +405,7 @@ Example output: ```javascript { range: { - test: { + fieldName: { gte: 500, lte: 1000, }, @@ -370,7 +413,108 @@ Example output: } ``` -TODO: -- All the other types! +You can read more about it in: + +- [Source code of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). +- [Unit tests of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). + +## Number Range Histogram Type + +The type `numberRangeHistogram` represents a number range with inclusive bounds. This type +returns feedback in the form of histogram and statistical data. +This type returns it's values in the `context` of the node, rather +than in the `results` node. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | +| `percentileInterval` | Number | Used to group the results based on how many of the records are within each one of the sections given by the interval, from the `min` value, up to the `max` value. | + +Some Notes: +1. An empty value as the upper boundary represents infinity. +2. An empty value as the lower boundary represents negative infinity. +3. Zero has to be respected as a boundary value. + +- [Source code of the type: numberRangeHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/numberRangeHistohgram.js). +- [Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/search-aggregations-bucket-histogram-aggregation.html). + +## Query Type + +Query represents a raw elasticsearch query_string. It's mostly used to +provide a simple sarch box on the user interface. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | +| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches (accepting values if they contain the given query, even if the casing doesn't match). | + +Example input: +```js +{ + type: 'query', + field: 'fieldName', + query: 'cable', + exact: true, +} +``` + +Example output: +```js +{ + query_string: { + query: 'cable', + default_operator: 'AND', + default_field: 'fieldName.exact', + analyzer: 'exact', + }, +} +``` + +You can read more about it in: + +- [Source code of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +- [Unit tests of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). + +## Text Type + +Text implements raw text analysis like starts with, ends with, etc. +These are generally regex queries. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | +| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches (accepting values if they contain the given query, even if the casing doesn't match). | + +Example input: +```js +{ + key: 'test', + type: 'text', + field: 'description', + join: 'any', + operator: 'contains', + values: ['laserjet', 'printer'], +} +``` + +Example output: +```js +{ + query_string: { + default_field: 'description', + default_operator: 'OR', + query: '"laserjet" "printer"', + }, +} +``` + +You can read more about it in: + +- [Source code of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/text.js). +- [Unit tests of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/text.js). [↪ Next: Mongo Example Types](mongo-example-types.md) From 27365ea226ddedd1659b765d7d6d39a67eee3e3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 5 Jun 2018 20:10:00 +0000 Subject: [PATCH 093/150] Mongo types too --- types/elasticsearch-example-types.md | 31 +++- types/mongo-example-types.md | 237 ++++++++++++++++++++++++++- 2 files changed, 258 insertions(+), 10 deletions(-) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index e619f7f..6b2c54c 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -16,6 +16,20 @@ fulfill you expectations. Our ElasticSearch types are the following ones: +## Results Type + +The `results` node is a node that if present, idicates to Contextre +that the queries and aggregations should finally run and retrieve all +the filtered values. It allows some customization properties, as +shown below: + +| Property Name | Type | Description | +| --- | --- | --- | +| `page` | Number | Current page of results. | +| `pageSize` | Number | Total results per page. | +| `sortField` | String | Field used to sort the results. | +| `sortDir` | String | Direction of sorting (`asc`, `desc`) | + ## Bool Type The bool type is intended to work as an ElasticSearch terms @@ -237,7 +251,7 @@ It requires only two fields: `field` and `value`. Example input: ```javascript { - type: 'bool', + type: 'exists', field: 'fieldName', value: true } @@ -486,15 +500,15 @@ These are generally regex queries. | Property Name | Type | Description | | --- | --- | --- | | `field` | String | The field we will be using to filter the results. | -| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | -| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches (accepting values if they contain the given query, even if the casing doesn't match). | +| `join` | String | Either `any`, `all` or `none`. | +| `operator` | String | . | +| `values` | Array | . | Example input: ```js { - key: 'test', type: 'text', - field: 'description', + field: 'fieldName', join: 'any', operator: 'contains', values: ['laserjet', 'printer'], @@ -505,7 +519,7 @@ Example output: ```js { query_string: { - default_field: 'description', + default_field: 'fieldName', default_operator: 'OR', query: '"laserjet" "printer"', }, @@ -517,4 +531,9 @@ You can read more about it in: - [Source code of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/text.js). - [Unit tests of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/text.js). +## Other ElasticSearch Example Types + +For more informaion about other available example types, please check: + + [↪ Next: Mongo Example Types](mongo-example-types.md) diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md index a7055ab..590e1ae 100644 --- a/types/mongo-example-types.md +++ b/types/mongo-example-types.md @@ -1,22 +1,251 @@ [↩ Parent: Table of Contents](../README.md) [↩ Previous: ElasticSearch Example Types](elasticsearch-example-types.md) -## Type +# MongoDb Example Types + +Most of our types have relevant components already written to +facilitate building search interfaces. As we progress over each one of +these types, we will show small examples of simple components written +to search with these types. We hope to provide enough information to +allow you to take as little as you need, or as much as you want, and +fulfill you expectations. + +Our MongoDb types are the following ones: + +## Results Type + +The `results` node is a node that if present, idicates to Contextre +that the queries and aggregations should finally run and retrieve all +the filtered values. It allows some customization properties, as +shown below: + +| Property Name | Type | Description | +| --- | --- | --- | +| `page` | Number | Current page of results. | +| `pageSize` | Number | Total results per page. | +| `sortField` | String | Field used to sort the results. | +| `sortDir` | String | Direction of sorting (`asc`, `desc`) | + +## Date Type + +The Date type is used to specify a range of dates that will be used to +filter the available results. This range of dates can be specified by +a string formatted date (`YYYY-MM-DD`) or by a small set of possible +humanly readable date ranges. Details follow: | Property Name | Type | Description | | --- | --- | --- | +| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | +| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | +| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | +| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | Example input: ```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25' +} ``` Example output: +```javascript +{ + fieldName: { + $gte: new Date('2016-04-25'), + }, +} +``` + +You can read more about it in: + +- [Source code of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). +- [Unit tests of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). +- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). +- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). + +## Exists Type + +The `exists` type is used to check wether some property exsits or not. +It requires only two fields: `field` and `value`. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The target field we want to check. | +| `value` | Boolean | the value we want to check. Normally true or false. | +Example input: ```javascript +{ + type: 'exists', + field: 'fieldName', + value: true +} ``` -- [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). -- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). -- Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). +Example output: + +```javascript +{ + $and: [ + { + fieldName: { + $exists: true, + $ne: '', + }, + }, + { + fieldName: { + $ne: null, + }, + }, + ], +} +``` + +You can read more about it in: + +- [Source code of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/exists.js). +- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/exists.js). +- MongoDb's [$exists](https://docs.mongodb.com/manual/reference/operator/query/exists/). + +## Facet Type + +The `facet` type represents a list of dynamic choices, e.g. a checkbox +list filter. We achieve this by running an ElasticSearch terms +aggregation. We provide a way to limit the number of results that you +will receive with the `size` property, so that large queries can +safely be used. For that same purpose, the property `optionsFilter` is +given, so that search queries can filter the results with a string. + +Facet returns it's values in the `context` of the node, rather than in +the `results` node. + +| Property | Type | Default | Description | +| ---- | ---- | ------- | ----------- | +| `field` | String | None, *required* | The field it's operating on. | +| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | +| `values` | Array (of strings) | `[]` | Already selected values. | +| `size` | Number | 10 | How many options to return. | + +Example input: +```javascript +{ + type: 'facet', + field: 'fieldName', + values: ['abc', '123'] +} +``` + +Example input with exclude: +```javascript +{ + type: 'facet', + field: 'fieldName', + mode: 'exclude', + values: ['abc', '123'], +} +``` + +You can read more about it in: + +- [Source code of the type: facet](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). + +## Number Type + +Number represents a number range with inclusive bounds. This type +provides the ability to determine the best range values based on +percentile interval and range threshold. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | + +Some Notes: +1. An empty value as the upper boundary represents infinity. +2. An empty value as the lower boundary represents negative infinity. +3. Zero has to be respected as a boundary value. +4. If findBestRange is true it will return the best min and max range. + +Example input: +```javascript +{ + type: 'number', + field: 'fieldName', + min: 500, + max: 1000, +} +``` + +Example output: + +```javascript +{ + fieldName: { + $gte: 500, + $lte: 1000 + } +} +``` + +You can read more about it in: + +- [Source code of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). +- [Unit tests of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). +- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). +- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). + +## Text Type + +Text implements raw text analysis like starts with, ends with, etc. +These are generally regex queries. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `join` | String | Either `any`, `all` or `none`. | +| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | +| `values` | Array | Array containing all the words that want to be used as inputs. | + +Example input: +```js +{ + type: 'text', + field: 'fieldName', + join: 'any', + operator: 'contains', + values: ['laserjet', 'printer'], +} +``` + +Example output: +```js +{ + $or: [{ + fieldName: { + $regex: 'laserjet', + $options: 'i' + } + }, { + fieldName: { + $regex: 'printer', + $options: 'i' + } + }] +} +``` + +You can read more about it in: + +- [Source code of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/text.js). +- [Unit tests of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/text.js). + +## Other MongoDb Example Types + +For more informaion about other available example types, please check: + [↪ Next: Available React Components for Types](react-components.md) From b1e8991cc0fc73a1cc02f4736c409a132211047f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 5 Jun 2018 20:11:50 +0000 Subject: [PATCH 094/150] more info on the es types --- types/elasticsearch-example-types.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md index 6b2c54c..08d6426 100644 --- a/types/elasticsearch-example-types.md +++ b/types/elasticsearch-example-types.md @@ -501,8 +501,8 @@ These are generally regex queries. | --- | --- | --- | | `field` | String | The field we will be using to filter the results. | | `join` | String | Either `any`, `all` or `none`. | -| `operator` | String | . | -| `values` | Array | . | +| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | +| `values` | Array | Array containing all the words that want to be used as inputs. | Example input: ```js From 5d6ab3a316ff589e93633eddbac0464f4d02a68d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Tue, 5 Jun 2018 20:38:09 +0000 Subject: [PATCH 095/150] Improved these files --- README.md | 2 + other-components/README.md | 5 +- other-components/general.md | 51 ++++++++++++++ other-components/layout.md | 133 ++++++++++++++++++++++++++++++++++++ 4 files changed, 190 insertions(+), 1 deletion(-) create mode 100644 other-components/general.md create mode 100644 other-components/layout.md diff --git a/README.md b/README.md index f57dd25..44bfd29 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,8 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Mongo Example Types](types/mongo-example-types.md) - [Available React Components for Types](types/react-components.md) - [Other Components](other-components/README.md) + - [Generally Useful Components](other-components/general.md) + - [Layout Components](other-components/layout.md) - [Managing State](managing-state/README.md) - [DIY State Management](managing-state/diy.md) - [MobX](managing-state/mobx.md) diff --git a/other-components/README.md b/other-components/README.md index 7cf1eb6..8305437 100644 --- a/other-components/README.md +++ b/other-components/README.md @@ -1,3 +1,6 @@ [↩ Parent: Table of Contents](../README.md) - # Other Components +# Other Components + +- [Generally Useful Components](other-components/general.md). +- [Layout Components](other-components/layout.md). diff --git a/other-components/general.md b/other-components/general.md new file mode 100644 index 0000000..634bee9 --- /dev/null +++ b/other-components/general.md @@ -0,0 +1,51 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Other Components](README.md) + +## Generally Useful Components + +### ContextureProvider + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### FilterAdder + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### FilterList + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +[↪ Next: Layout Components](layout.md) diff --git a/other-components/layout.md b/other-components/layout.md new file mode 100644 index 0000000..d91404b --- /dev/null +++ b/other-components/layout.md @@ -0,0 +1,133 @@ +[↩ Parent: Table of Contents](../README.md) +[↩ Previous: Generally Useful Components](general.md) + +## Layout Components + +### Awaiter + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### BarChart +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### Dynamic +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### Flex +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### Modal +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### Pickers +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### Popover +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### SpacedList +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +### TextHighlight +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minumum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/') +// ... +// Later, on your render function, or where you put your components: + +``` + +[↪ Next: Managing State](../managing-state/README.md) From d367a56018b2a4c5a70c5f88e50cf82f33236814 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 6 Jun 2018 00:44:35 +0000 Subject: [PATCH 096/150] Cleaning these up --- other-components/general.md | 76 +++++++++++++------ other-components/layout.md | 147 ++++++++++++++++-------------------- types/react-components.md | 2 +- 3 files changed, 123 insertions(+), 102 deletions(-) diff --git a/other-components/general.md b/other-components/general.md index 634bee9..4b0c3ec 100644 --- a/other-components/general.md +++ b/other-components/general.md @@ -5,47 +5,81 @@ ### ContextureProvider -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +This component is a magic tool to make Contexture search interfaces without +having to write the tree separated. With this component, you can pass the tree +directly as a plain object, and any children you pass will receive the `tree` +property directly. -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/') -// ... -// Later, on your render function, or where you put your components: - -``` - -### FilterAdder +This component receives: | Property Name | Type | Required | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +| `types` | Object | Yes | Your client-side types, such as the default types of the Contexture Client. | +| `service` | Object | Yes | The search function that sends the DSL to the initialized Contexture Core. | +| `nodeKey` | Object | Yes | Key for the root node. Defaults with `root`. | +| `...props` | Any other property (children excluded) | Yes | Any other property that you might want to send to the initialization of the Contexture Client. | + +**Note:** Any of the Contexture React Example Types components automatically +add the nodes to the tree if the referenced node is not present. _This is +an experimental feature_. Here's how you write your component: ```javascript -let Number = require('contexture-react/dist/') +let ContextureProvider = require('contexture-react/dist/ContextureProvider') +let ContextureClient = require('contexture-client') +let types = ContextureClient.exampleTypes +let service = async search => ({ + data: await postData('/sarch', { search }) +}) // ... // Later, on your render function, or where you put your components: - + + + ``` +- [Source code of the ContextureProvider component](https://github.com/smartprocure/contexture-react/blob/master/src/ContextureProvider.js). + ### FilterList +A component that tries to automatically render the specific type components of +the children of a node. + | Property Name | Type | Required | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +| `node` | Object | Yes | Node of the Contexture Tree where the children that want to be rendered are. | +| `exampleTypes` | Object | No | Object which a key and a component per type. Defaults in the available example types components in contexture-elasticsearch. | +| `fields` | Object | Yes | Object which a key and an object with at least a `label` string property, used to indicate the label of each one of the filters. | Here's how you write your component: ```javascript -let Number = require('contexture-react/dist/') +let FilterList = require('contexture-react/dist/FilterList') +let ContextureClient = require('contexture-client') +let tree = ContextureClient({ + key: 'root', + type: 'group', + schema: 'mySchema', + children: [{ + key: 'query', + type: 'query', + field: 'myFieldName', + query: 'Something' + }] +}) // ... // Later, on your render function, or where you put your components: - + ``` +- [Source code of the FilterList component](https://github.com/smartprocure/contexture-react/blob/master/src/FilterList.js). + +Up next, components only useful for helping on building the layout. + [↪ Next: Layout Components](layout.md) diff --git a/other-components/layout.md b/other-components/layout.md index d91404b..9fb22b1 100644 --- a/other-components/layout.md +++ b/other-components/layout.md @@ -5,129 +5,116 @@ ### Awaiter +Renders a loading indicator until a Promise is resolved. It will ender +an exception (currently just `Ooops...`) if the Promise fails. if the +Promise passes, the children are rendered. + | Property Name | Type | Required | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +| `promise` | Promise | Yes | Minumum number for the range. | Here's how you write your component: ```javascript -let Number = require('contexture-react/dist/') +let Awaiter = require('contexture-react/dist/layout/Awaiter') +let promiseMaker = () => new Promise((resolve, reject)) /* ... */ resolve()) // ... // Later, on your render function, or where you put your components: - + +
My Crazy Children
+ ``` +- [Source code of the Awaiter component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/Awaiter.js). + ### BarChart -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/') -// ... -// Later, on your render function, or where you put your components: - -``` +Allows you to build your own bar charts (besides just relying on +the DateHistogram component). -### Dynamic -| Property Name | Type | Required | Description | +| Property Name | Type | Default Value | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +| `valueField` | String | `''` | The field of each record where the value used for the height of each barnis located. | +| `categoryField` | String | `''` | The field of each record where the label of each bar is located. | +| `data` | Array (of Objects with both the `valueField` and the `categoryField`) | `[]` | The data that is going to be used to build the chart. | +| `height` | Number | `100` | Specifies the max height of the whole chart. | +| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | +| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | +| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | Here's how you write your component: ```javascript -let Number = require('contexture-react/dist/') +let BarChart = require('contexture-react/dist/layout/BarChart') // ... // Later, on your render function, or where you put your components: - + ``` -### Flex -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | - -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/') -// ... -// Later, on your render function, or where you put your components: - -``` +- [Source code of the BarChart component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/BarChart.js). -### Modal -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +### SpacedList -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/') -// ... -// Later, on your render function, or where you put your components: - -``` +Wraps every children in a div with a given style. Useful for (as the +name portrays) making a spaced list. -### Pickers | Property Name | Type | Required | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +| `style` | Object | No | The style that will be applied to each div. Defaults in `{ marginBottom: '25px' }`. | Here's how you write your component: ```javascript -let Number = require('contexture-react/dist/') +let SpacedList = require('contexture-react/dist/layout/SpacedList') // ... // Later, on your render function, or where you put your components: - + +

Hi

+ + ``` -### Popover -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +- [Source code of the SpacedList component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/SpacedList.js). -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/') -// ... -// Later, on your render function, or where you put your components: - -``` +### TextHighlight + +Used to highlight content within a text basef on a pattern. -### SpacedList | Property Name | Type | Required | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +| `pattern` | String | No | RegExp pattern used to find matching content. | +| `text` | String | Yes | Text used as the target of the matches. | +| `Wrap` | Component | No | Component used to wrap each one of the +matched. Defaults to `i`. | Here's how you write your component: ```javascript -let Number = require('contexture-react/dist/') +let TextHighlight = require('contexture-react/dist/layout/TextHighlight') // ... // Later, on your render function, or where you put your components: - + ``` -### TextHighlight -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | -| `max` | Number | No | Maximum number for the range. | +- [Source code of the TextHighlight component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/TextHighlight.js). -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/') -// ... -// Later, on your render function, or where you put your components: - -``` +Up next, our recommendations on state management. [↪ Next: Managing State](../managing-state/README.md) diff --git a/types/react-components.md b/types/react-components.md index 5326ac3..2804406 100644 --- a/types/react-components.md +++ b/types/react-components.md @@ -251,7 +251,7 @@ Here is the list of properties that this component expects to have on the node: | Property Name | Type | Required | Description | | --- | --- | --- | --- | -| `min` | Number | No | Minumum number for the range. | +| `min` | Number | No | Minimum number for the range. | | `max` | Number | No | Maximum number for the range. | **Note:** The properties present in the search tree that aren't used by the node From 5974f1ecd82ef3f755cd50653e06df6111396bd1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Rodr=C3=ADguez?= Date: Wed, 6 Jun 2018 00:46:34 +0000 Subject: [PATCH 097/150] Fixed links --- other-components/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/other-components/README.md b/other-components/README.md index 8305437..778d7fb 100644 --- a/other-components/README.md +++ b/other-components/README.md @@ -2,5 +2,5 @@ # Other Components -- [Generally Useful Components](other-components/general.md). -- [Layout Components](other-components/layout.md). +- [Generally Useful Components](general.md). +- [Layout Components](layout.md). From 2b2592ee80fea9ff05fc2676ff2f9611e74f70a6 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:07:01 -0500 Subject: [PATCH 098/150] add index.html and CNAME --- CNAME | 1 + index.html | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 CNAME create mode 100644 index.html diff --git a/CNAME b/CNAME new file mode 100644 index 0000000..f5d7d38 --- /dev/null +++ b/CNAME @@ -0,0 +1 @@ +docs.contexture.site diff --git a/index.html b/index.html new file mode 100644 index 0000000..4aabe15 --- /dev/null +++ b/index.html @@ -0,0 +1,67 @@ + + + + + + + + + Contexture + + + + + + + + + + + + + + + + + + + +
+
+

Contexture

+ +
+
+ + +
+
+ +
+
+
+
+
+
+ + + From 6825ea15f8ec1b6796b4c683a63685f13a8254cf Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:08:48 -0500 Subject: [PATCH 099/150] add docs dir --- index.html => docs/index.html | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename index.html => docs/index.html (100%) diff --git a/index.html b/docs/index.html similarity index 100% rename from index.html rename to docs/index.html From 55110faf9706899de0bb56f7ee982a2359723113 Mon Sep 17 00:00:00 2001 From: ltchris <29695350+ltchris@users.noreply.github.com> Date: Sun, 8 Mar 2020 17:09:34 -0500 Subject: [PATCH 100/150] Create CNAME --- docs/CNAME | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/CNAME diff --git a/docs/CNAME b/docs/CNAME new file mode 100644 index 0000000..d2b8e1f --- /dev/null +++ b/docs/CNAME @@ -0,0 +1 @@ +docs.contexture.site \ No newline at end of file From 450a555c0547ff38aaa532d4fdd3f3ad09c0a7b5 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:09:54 -0500 Subject: [PATCH 101/150] remove root CNAME --- CNAME | 1 - 1 file changed, 1 deletion(-) delete mode 100644 CNAME diff --git a/CNAME b/CNAME deleted file mode 100644 index f5d7d38..0000000 --- a/CNAME +++ /dev/null @@ -1 +0,0 @@ -docs.contexture.site From b1654bc5de53f63b83ed99e6f69c5db5b4c1128c Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:24:32 -0500 Subject: [PATCH 102/150] add formatted sections --- README.md | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/README.md b/README.md index 44bfd29..d7cae66 100644 --- a/README.md +++ b/README.md @@ -86,3 +86,90 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. - [Examples](examples/README.md) - [Contributing Guide](contributing-guide/README.md) - [License](LICENSE) + +## About Contexture + +### What is Contexture +### Glossary of Terms +### Map of Repos +### Brief History +### Alternatives & Benchmarks + +## Getting Started + +### Project Setup +#### Install Node 9 and NPM +#### Install Contexture +#### Install Contexture Client +#### Install Contexture ElasticSearch +#### Install Contexture Mongo +#### Install Contexture React + +### A First Contexture Script +### Connecting to Elasticsearch & MongoDB +Connecting to ElasticSearch +Connecting to MongoDB +Connecting to Other Databases +Simple Search Box +Your First Filter +Discovering the Database +IMDB Index +Querying +Contexture DSL +Interactive Queries +Contexture Client +Initializing the Contexture Client +Context Tree +Tree and getNode +Mutate, Add and Remove +Introduction to Reactors +Types and Type Components +DIY Types +How to Write a Provider Type +How to Write a Client Type +How to Write a UI Component for a Type +The Example Types +ElasticSearch Example Types +Mongo Example Types +Available React Components for Types +Other Components +Generally Useful Components +Layout Components +Managing State +DIY State Management +MobX +Redux (Coming Soon) +Theming +Recommendations +Architecture +Client vs Server +Type definitions +Scaling +Server Side Searches +Searching On an Endpoint +Caching +Client Side Searches +Click to Search +Real Time Searches +Cascading +Under the Hood +Design Principles +Contexture Core +Default Export +The Algorithm +Utility Functions +Contexture Providers +Contexture ElasticSearch +Contexture Mongo +Building Your Own Provider +Contexture Client +State Properties +Lenses +Reactors in Detail +Serialization +Traversals +Contexture React +Examples +Contributing Guide +License + From f8344550ec1643f7990ca8ff90fe1bf8417b0c49 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:25:42 -0500 Subject: [PATCH 103/150] remove TOC --- README.md | 84 ------------------------------------------------------- 1 file changed, 84 deletions(-) diff --git a/README.md b/README.md index d7cae66..e283318 100644 --- a/README.md +++ b/README.md @@ -3,90 +3,6 @@ This repo is designed to host the documentation of Contexture as a whole. Everythin we currently have here is **WORK IN PROGRESS**. -## Table of Contents - -- [About Contexture](about/README.md) - - [What is Contexture](about/what-is-contexture.md) - - [Glossary of Terms](about/glossary-of-terms.md) - - [Map of Repos](about/map-of-repos.md) - - [Brief History](about/brief-history.md) - - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) -- [Getting Started](getting-started/README.md) - - [Project Setup](getting-started/setup.md) - - [Install Node 9 and NPM](getting-started/setup.md#installing-node-9-and-npm) - - [Install Contexture](getting-started/setup.md#installing-contexture) - - [Install Contexture Client](getting-started/setup.md#installing-contexture-client) - - [Install Contexture ElasticSearch](getting-started/setup.md#installing-contexture-elasticsearch) - - [Install Contexture Mongo](getting-started/setup.md#installing-contexture-mongo) - - [Install Contexture React](getting-started/setup.md#installing-contexture-react) - - [A First Contexture Script](getting-started/first-script.md) - - [Connecting to Elasticsearch & MongoDB](getting-started/connecting.md) - - [Connecting to ElasticSearch](getting-started/connecting.md#connecting-to-elasticsearch.md) - - [Connecting to MongoDB](getting-started/connecting.md#connecting-to-mongodb.md) - - [Connecting to Other Databases](getting-started/connecting-other-databases.md) - - [Simple Search Box](getting-started/simple-search-box.md) - - [Your First Filter](getting-started/your-first-filter.md) - - [Discovering the Database](getting-started/discovering-the-database.md) - - [IMDB Index](getting-started/imdb-example.md) -- [Querying](querying/README.md) - - [Contexture DSL](querying/contexture-dsl.md) - - [Interactive Queries](querying/interactive-queries/README.md) - - [Contexture Client](querying/interactive-queries/contexture-client.md) - - [Initializing the Contexture Client](querying/interactive-queries/contexture-client.md#initializing-the-contexture-client) - - [Context Tree](querying/interactive-queries/contexture-client.md#context-tree) - - [Tree and getNode](contexture-client.md#tree-and-getnode) - - [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) - - [Introduction to Reactors](querying/interactive-queries/reactors.md) -- [Types and Type Components](types/README.md) - - [DIY Types](types/diy-types.md) - - [How to Write a Provider Type](types/diy-types.md#how-to-wite-a-provider-type) - - [How to Write a Client Type](types/diy-types.md#how-to-wite-a-client-type) - - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) - - [The Example Types](types/diy-types.md#the-example-types) - - [ElasticSearch Example Types](types/elasticsearch-example-types.md) - - [Mongo Example Types](types/mongo-example-types.md) - - [Available React Components for Types](types/react-components.md) -- [Other Components](other-components/README.md) - - [Generally Useful Components](other-components/general.md) - - [Layout Components](other-components/layout.md) -- [Managing State](managing-state/README.md) - - [DIY State Management](managing-state/diy.md) - - [MobX](managing-state/mobx.md) - - [Redux (Coming Soon)](managing-state/redux.md) -- [Theming](theming/README.md) -- [Recommendations](recommendations/README.md) - - [Architecture](recommendations/architecture.md) - - [Client vs Server](recommendations/architecture.md#client-vs-server) - - [Type definitions](recommendations/architecture.md#type-definitions) - - [Scaling](recommendations/architecture.md#scaling) - - [Server Side Searches](recommendations/server-side-searches.md) - - [Searching On an Endpoint](recommendations/server-side-searches.md#searching-on-an-endpoint) - - [Caching](recommendations/server-side-searches.md#caching) - - [Client Side Searches](recommendations/client-side-searches.md) - - [Click to Search](recommendations/client-side-searches.md#click-to-search) - - [Real Time Searches](recommendations/client-side-searches.md#real-time-searches) - - [Cascading](recommendations/client-side-searches.md#cascading) -- [Under the Hood](under-the-hood/README.md) - - [Design Principles](under-the-hood/design-principles.md) - - [Contexture Core](under-the-hood/contexture-core.md) - - [Default Export](under-the-hood/contexture-core.md#default-export) - - [The Algorithm](under-the-hood/contexture-core.md#the-algorithm) - - [Utility Functions](under-the-hood/contexture-core.md#utility-functions) - - [Contexture Providers](under-the-hood/contexture-providers/README.md) - - [Contexture ElasticSearch](under-the-hood/contexture-providers/) - - [Contexture Mongo](under-the-hood/contexture-providers/) - - [Building Your Own Provider](under-the-hood/contexture-providers/building-your-own-provider.md) - - [Contexture Client](under-the-hood/contexture-client.md) - - [State Properties](under-the-hood/contexture-client.md#state-properties) - - [Lenses](under-the-hood/contexture-client.md#lenses) - - [Reactors in Detail](under-the-hood/contexture-client.md#reactors-in-detail) - - [Serialization](under-the-hood/contexture-client.md#serialization) - - [Traversals](under-the-hood/contexture-client.md#traversals) - - [Contexture React](under-the-hood/contexture-react.md) -- [Examples](examples/README.md) -- [Contributing Guide](contributing-guide/README.md) -- [License](LICENSE) - ## About Contexture ### What is Contexture From 2e8041533a0e4e72065598a7589d34d34e4953e1 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:32:26 -0500 Subject: [PATCH 104/150] fix all sections formatting --- README.md | 142 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 74 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index e283318..5727eae 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,6 @@ This repo is designed to host the documentation of Contexture as a whole. Everythin we currently have here is **WORK IN PROGRESS**. ## About Contexture - ### What is Contexture ### Glossary of Terms ### Map of Repos @@ -12,7 +11,6 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. ### Alternatives & Benchmarks ## Getting Started - ### Project Setup #### Install Node 9 and NPM #### Install Contexture @@ -20,72 +18,80 @@ whole. Everythin we currently have here is **WORK IN PROGRESS**. #### Install Contexture ElasticSearch #### Install Contexture Mongo #### Install Contexture React - ### A First Contexture Script ### Connecting to Elasticsearch & MongoDB -Connecting to ElasticSearch -Connecting to MongoDB -Connecting to Other Databases -Simple Search Box -Your First Filter -Discovering the Database -IMDB Index -Querying -Contexture DSL -Interactive Queries -Contexture Client -Initializing the Contexture Client -Context Tree -Tree and getNode -Mutate, Add and Remove -Introduction to Reactors -Types and Type Components -DIY Types -How to Write a Provider Type -How to Write a Client Type -How to Write a UI Component for a Type -The Example Types -ElasticSearch Example Types -Mongo Example Types -Available React Components for Types -Other Components -Generally Useful Components -Layout Components -Managing State -DIY State Management -MobX -Redux (Coming Soon) -Theming -Recommendations -Architecture -Client vs Server -Type definitions -Scaling -Server Side Searches -Searching On an Endpoint -Caching -Client Side Searches -Click to Search -Real Time Searches -Cascading -Under the Hood -Design Principles -Contexture Core -Default Export -The Algorithm -Utility Functions -Contexture Providers -Contexture ElasticSearch -Contexture Mongo -Building Your Own Provider -Contexture Client -State Properties -Lenses -Reactors in Detail -Serialization -Traversals -Contexture React -Examples -Contributing Guide -License +#### Connecting to ElasticSearch +#### Connecting to MongoDB +### Connecting to Other Databases +### Simple Search Box +### Your First Filter +### Discovering the Database +### IMDB Index + +## Querying +### Contexture DSL +### Interactive Queries +#### Contexture Client +##### Initializing the Contexture Client +##### Context Tree +##### Tree and getNode +##### Mutate, Add and Remove +#### Introduction to Reactors + +## Types and Type Components +### DIY Types +#### How to Write a Provider Type +#### How to Write a Client Type +#### How to Write a UI Component for a Type +#### The Example Types +### ElasticSearch Example Types +### Mongo Example Types +### Available React Components for Types + +## Other Components +### Generally Useful Components +### Layout Components + +## Managing State +### DIY State Management +### MobX +### Redux (Coming Soon) + +## Theming + +## Recommendations +### Architecture +#### Client vs Server +#### Type definitions +#### Scaling +### Server Side Searches +#### Searching On an Endpoint +#### Caching +### Client Side Searches +#### Click to Search +#### Real Time Searches +#### Cascading + +## Under the Hood +### Design Principles +### Contexture Core +#### Default Export +#### The Algorithm +#### Utility Functions +### Contexture Providers +#### Contexture ElasticSearch +#### Contexture Mongo +#### Building Your Own Provider +### Contexture Client +#### State Properties +#### Lenses +#### Reactors in Detail +#### Serialization +#### Traversals +### Contexture React + +## Examples + +## Contributing Guide +## License From ad6eca474572be1c776e509c834d1d84582f2b39 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:38:11 -0500 Subject: [PATCH 105/150] remove other doc directories --- about/README.md | 10 - about/alternatives-benchmarks.md | 8 - about/brief-history.md | 8 - about/glossary-of-terms.md | 92 --- about/map-of-repos.md | 69 --- about/what-is-contexture.md | 53 -- getting-started/README.md | 21 - getting-started/connecting-other-databases.md | 17 - getting-started/connecting.md | 323 ----------- getting-started/discovering-the-database.md | 121 ---- getting-started/first-script.md | 202 ------- getting-started/imdb-example.md | 73 --- getting-started/setup.md | 74 --- getting-started/simple-search-box.md | 240 -------- getting-started/your-first-filter.md | 74 --- managing-state/README.md | 1 - other-components/README.md | 6 - other-components/general.md | 85 --- other-components/layout.md | 120 ---- querying/README.md | 11 - querying/contexture-dsl.md | 58 -- querying/interactive-queries/README.md | 13 - .../interactive-queries/contexture-client.md | 179 ------ querying/interactive-queries/reactors.md | 44 -- types/README.md | 13 - types/diy-types.md | 182 ------ types/elasticsearch-example-types.md | 539 ------------------ types/mongo-example-types.md | 251 -------- types/react-components.md | 401 ------------- under-the-hood/README.md | 21 - under-the-hood/contexture-client.md | 1 - under-the-hood/contexture-core.md | 60 -- .../building-your-own-provider.md | 92 --- .../contexture-elasticsearch.md | 1 - .../contexture-providers/contexture-mongo.md | 1 - under-the-hood/contexture-react.md | 1 - under-the-hood/design-principles.md | 25 - 37 files changed, 3490 deletions(-) delete mode 100644 about/README.md delete mode 100644 about/alternatives-benchmarks.md delete mode 100644 about/brief-history.md delete mode 100644 about/glossary-of-terms.md delete mode 100644 about/map-of-repos.md delete mode 100644 about/what-is-contexture.md delete mode 100644 getting-started/README.md delete mode 100644 getting-started/connecting-other-databases.md delete mode 100644 getting-started/connecting.md delete mode 100644 getting-started/discovering-the-database.md delete mode 100644 getting-started/first-script.md delete mode 100644 getting-started/imdb-example.md delete mode 100644 getting-started/setup.md delete mode 100644 getting-started/simple-search-box.md delete mode 100644 getting-started/your-first-filter.md delete mode 100644 managing-state/README.md delete mode 100644 other-components/README.md delete mode 100644 other-components/general.md delete mode 100644 other-components/layout.md delete mode 100644 querying/README.md delete mode 100644 querying/contexture-dsl.md delete mode 100644 querying/interactive-queries/README.md delete mode 100644 querying/interactive-queries/contexture-client.md delete mode 100644 querying/interactive-queries/reactors.md delete mode 100644 types/README.md delete mode 100644 types/diy-types.md delete mode 100644 types/elasticsearch-example-types.md delete mode 100644 types/mongo-example-types.md delete mode 100644 types/react-components.md delete mode 100644 under-the-hood/README.md delete mode 100644 under-the-hood/contexture-client.md delete mode 100644 under-the-hood/contexture-core.md delete mode 100644 under-the-hood/contexture-providers/building-your-own-provider.md delete mode 100644 under-the-hood/contexture-providers/contexture-elasticsearch.md delete mode 100644 under-the-hood/contexture-providers/contexture-mongo.md delete mode 100644 under-the-hood/contexture-react.md delete mode 100644 under-the-hood/design-principles.md diff --git a/about/README.md b/about/README.md deleted file mode 100644 index 49cdb6c..0000000 --- a/about/README.md +++ /dev/null @@ -1,10 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - -# About Contexture - -Table of Contents: -- [What is Contexture](what-is-contexture.md) -- [Glossary of Terms](glossary-of-terms.md) -- [Map of Repos](map-of-repos.md) -- [Brief History](brief-history.md) -- [Alternatives & Benchmarks](alternatives-benchmarks.md) diff --git a/about/alternatives-benchmarks.md b/about/alternatives-benchmarks.md deleted file mode 100644 index cb4f186..0000000 --- a/about/alternatives-benchmarks.md +++ /dev/null @@ -1,8 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Brief History](brief-history.md) - -# Alternatives & Benchmarks - - TODO - -[↪ Next: Getting Started](../getting-started/README.md) diff --git a/about/brief-history.md b/about/brief-history.md deleted file mode 100644 index fdd3b2f..0000000 --- a/about/brief-history.md +++ /dev/null @@ -1,8 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Map of Repos](map-of-repos.md) - -# Brief History - - TODO - -[↪ Next: Alternatives & Benchmarks](alternatives-benchmarks.md) diff --git a/about/glossary-of-terms.md b/about/glossary-of-terms.md deleted file mode 100644 index c5d8788..0000000 --- a/about/glossary-of-terms.md +++ /dev/null @@ -1,92 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: What is Contexture](what-is-contexture.md) - -# Glossary of Terms - -## Contexture - -> NOUN -> 1. The fact or manner of being woven or linked together to form a -> connected whole. -> 1.1. A mass of things interwoven together; a fabric. -> 1.2. The putting together of words and sentences in connected -> composition; the construct of a text. -> 1.3. A connected literary structure; a continuous text. - -[Source](https://en.oxforddictionaries.com/definition/us/contexture). - -## Domain-Specific Language (DSL) - -> A domain-specific language (DSL) is a computer language specialized -> to a particular application domain. This is in contrast to a -> general-purpose language (GPL), which is broadly applicable across -> domains. - -[Source](https://en.wikipedia.org/wiki/Domain-specific_language). - -## Search Query - -According to Wikipedia, `Search Query`, in the context of the `web`, refers to: - -> ...a query that a user enters into a web search engine to satisfy his or her -> information needs. Web search queries are distinctive in that they are often -> plain text or hypertext with optional search-directives (such as "and"/"or" -> with "-" to exclude). They vary greatly from standard query languages, which -> are governed by strict syntax rules as command languages with keyword or -> positional parameters. - -[Source](https://en.wikipedia.org/wiki/Web_search_query). - -## Faceted Search - -> Faceted search, also known as faceted navigation or faceted browsing, is a -> technique used by eCommerce brands to help users analyze, organize, and -> filter large sets of product inventory based on filters such as size, color, -> price, and brand. - -[Source](https://www.dynamicyield.com/glossary/faceted-search/). - -> Faceted search, also called faceted navigation or faceted browsing, is a -> technique for accessing information organized according to a faceted -> classification system, allowing users to explore a collection of information -> by applying multiple filters. A faceted classification system classifies each -> information element along multiple explicit dimensions, called facets, -> enabling the classifications to be accessed and ordered in multiple ways -> rather than in a single, pre-determined, taxonomic order. - -[Source](https://en.wikipedia.org/wiki/Faceted_search). - -## Search Filter - -> An extension of faceted search, a search filter is a specific product -> attribute a visitor can use to refine the search results of a particular -> category listing, e.g. by size, color, price, or brand. Multiple filters may -> be applied to take a broad range of products and refine them into a more -> narrow selection, allowing the end user to retrieve the most relevant search -> results based on the criteria they’ve selected. - -[Source](https://www.dynamicyield.com/glossary/search-filter/). - -## Configuration Based Development - -> The difference between configuration-driven development and model-driven -> development is that the former is not restricted to the model of the code -> such as classes, fields, and relationships. Configuration-driven development -> (CCD) encompasses anything that can be configured within your application. -> For example, if your architecture dictates that particular business rules -> must be applied consistently across your application, you can use -> configuration files to configure and apply those rules. - -[Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). - -## Functional Logic Programming - -> Functional logic programming is the combination, in a single programming -> language, of the paradigms of functional programming (including higher-order -> programming) and logic programming (nondeterministic programming, -> unification). This style of programming is embodied by various programming -> languages, including Curry and Mercury. - -[Source](https://en.wikipedia.org/wiki/Functional_logic_programming). - -[↪ Next: Map of Repos](map-of-repos.md) diff --git a/about/map-of-repos.md b/about/map-of-repos.md deleted file mode 100644 index a3e7523..0000000 --- a/about/map-of-repos.md +++ /dev/null @@ -1,69 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Glossary of Terms](glossary-of-terms.md) - -# Map of Repos - -The Contexture framework comes to life through a list of repositories -that individually specialize in some needed layer for our -architecture. We will be using most (if not all) of these projects in -our upcoming pages. Let's explore the repos we have so far: - -## Contexture Core - -[github.com/smartprocure/contexture](https://github.com/smartprocure/contexture) -is where our main DSL processor lives. It is a very small layer that -ties everything together. This one receives the information about the -different search representations, about the databases involved, and -the DSL, then outputs the search results respective to each one of the -queries described in a copy of the received DSL. - -You can read more about the core here: -- [In the repository](https://github.com/smartprocure/contexture). -- In our [docs about querying (Contexture Core section)](../querying/contexture-core.md). -- Or with greater detail in our [under the hood docs (Contexture Core section)](../under-the-hood/contexture-core.md). - -## Contexture Providers - -The Contexture Providers are the interfacing layer that ties the -Contexture DSL to the targeted databases. So far, we have only two -open source providers: - -- [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch), - and -- [contexture-mongo](https://github.com/smartprocure/contexture-mongo). - -If you are planning to use either ElasticSearch or MongoDB for your -project, the best way to get started is to use those repositories. -However, if you need to use Contexture with another database, you will -need to implement the provider yourself. We're looking for code -contributors, so please don't feel limited to the current available -tools. Help us grow together! - -You can read more about Contexture Providers here: -- In our [docs about querying (Available Providers section)](../querying/available-providers.md). -- In with greater detail in our [under the hood docs (Contexture Providers section)](../under-the-hood/contexture-providers/README.md). - -## Contexture Client - -The Contexture Client is responsible for triggering behaviors on the -search interfaces by knowing what causes changes in one or more -elements of the search tree. It is the key piece of technology that -allows our search interfaces to work in real time. - -You can read more about the Contexture Client here: -- [In the repository](https://github.com/smartprocure/contexture-client). -- In our [docs about querying (Contexture Client section)](../interactive-queries/contexture-client.md). -- Or with greater detail in our [under the hood docs (Contexture Client section)](../under-the-hood/contexture-client.md). - -## Contexture React - -The Contexture React repository holds a list of components that -facilitate building search interfaces. They are mainly graphical -representations of the types that exist on our Contexture Providers. - -You can read more about the Contexture React: -- [In the repository](https://github.com/smartprocure/contexture-client). -- In our guide for the [Available React Components for Types](./types/react-components.md). -- Or in greater detail in our [under the hood docs (Contexture React section)](under-the-hood/contexture-react.md). - -[↪ Next: Brief History](brief-history.md) diff --git a/about/what-is-contexture.md b/about/what-is-contexture.md deleted file mode 100644 index 71cbd9f..0000000 --- a/about/what-is-contexture.md +++ /dev/null @@ -1,53 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: About Contexture](README.md) - -# What Is Contexture - -People of the Internet, here we officialy introduce you to -`contexture`, our framework for building search interfaces. - -This framework is carefully designed to be a generic solution for a -universe of unlimited possible search interfaces. We've started with a -minimal set of repositories that are representative of tools that -empower our business, but are intended to be merely examples. If -anything, our approaches are only use cases, for the potential of -this tool is ultimately yours to take. - -A quick search over the Internet would reveal that the word -`contexture` means: `the fact or manner of being woven or linked -together to form a connected whole` and also `the putting together of -words and sentences in connected composition; the construction of a -text`. - -Picking `contexture` as the name for this project means that we are -trying to expose not only our ultimate intentions, but also more or -less how the system is built. The way our projects work is by a DSL -that is used to gather different intended search inputs, each one -representing some useful abstraction of a search filter (like an input -where you can write a word to be searched, or another where you can -filter the search results by one or more options), then using the -values to process a DSL that will end up retrieving values from one or -more different databases, then returning these values on the -respective sections of the DSL, so that each result can update each -one of the components of the user interface. A more detailed -description is visible in the following diagram. - -

- -The canonical example of a Contexture Node is faceted search, where -you have a checkbox list that is both a filter (in the sense that it -restricts results based on the checked values) and an aggregation -(which shows the top n values that can be checked). Contexture allows -them to be nested in advanced searches with boolean joins like -`and`/`or`/`not`. - -

- -This thought process will become more clear as we progress through the -docs. Hopefully, some pages later it will be easy to grasp how we -provide a new perspective on building search interfaces, and perhaps -even how you can use it to power up your business, just like we have -been doing for almost a decade. - -[↪ Next: Glossary of Terms](glossary-of-terms.md) -[↪ Or perhaps you want to jump to: Map of Repos](map-of-repos.md) diff --git a/getting-started/README.md b/getting-started/README.md deleted file mode 100644 index 47dbb00..0000000 --- a/getting-started/README.md +++ /dev/null @@ -1,21 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - -# Getting Started - -Table of Contents: -- [Project Setup](setup.md) - - [Install Node 9 and NPM](setup.md#installing-node-9-and-npm) - - [Install Contexture](setup.md#installing-contexture) - - [Install Contexture Client](setup.md#installing-contexture-client) - - [Install Contexture ElasticSearch](setup.md#installing-contexture-elasticsearch) - - [Install Contexture Mongo](setup.md#installing-contexture-mongo) - - [Install Contexture React](setup.md#installing-contexture-react) -- [A First Contexture Script](first-script.md) -- [Connecting to Elasticsearch & MongoDB](connecting.md) - - [Connecting to ElasticSearch](connecting.md#connecting-to-elasticsearch.md) - - [Connecting to MongoDB](connecting.md#connecting-to-mongodb.md) -- [Connecting to Other Databases](connecting-other-databases.md) -- [Simple Search Box](simple-search-box.md) -- [Your First Filter](your-first-filter.md) -- [Discovering the Database](discovering-the-database.md) -- [IMDB Index](imdb-example.md) diff --git a/getting-started/connecting-other-databases.md b/getting-started/connecting-other-databases.md deleted file mode 100644 index 091a243..0000000 --- a/getting-started/connecting-other-databases.md +++ /dev/null @@ -1,17 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Connecting to Elasticsearch & MongoDB](connecting.md) - -## Connecting to Other Databases - -Contexture depends on it's providers to be able to know how to -translate from the Contexture DSL to the specific DSL that each -database needs. Because of this, to connect to other databases you -will need to create a new Provider. - -If you want to build your own provider, please follow this detailed -guide: [Building Your Own -Provider](../under-the-hood/contexture-providers/building-your-own-provider.md). -Beware that it might get into the core of contexture, which is not -needed if you can take advantage of the existing providers. - -[↪ Next: Simple Search Box](simple-search-box.md) diff --git a/getting-started/connecting.md b/getting-started/connecting.md deleted file mode 100644 index b96dcc6..0000000 --- a/getting-started/connecting.md +++ /dev/null @@ -1,323 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Project Setup](setup.md) - -## Connecting to Elasticsearch & MongoDB - -Our primary use case for advanced search interfaces is to query data -that we store on ElasticSearch and MongoDB. Because of that, we -provide two contexture libraries for those databases. In this page -we'll examine how to use these repositories to connect to existing -Mongo databases and ElasticSearch indexes. - -### Connecting to ElasticSearch - -The followng code example shows how to connect to ElasticSearch: - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-elasticsearch') -let types = require('contexture-elasticsearch/types') -let elasticsearch = require('elasticsearch') -let AgentKeepAlive = require('agentkeepalive') - -let elasticClient = null -let getClient = () => { - if (elasticClient) return elasticClient - elasticClient = elasticsearch.Client({ - minSockets: 1, - maxSockets: 20, - keepAlive: true, - createNodeAgent: (connection, config) => - new AgentKeepAlive(connection.makeAgentConfig(config)) - }) - return elasticClient -} - -let schemas = { - yourCustomSchemaName: { - elasticsearch: { - index: 'SomeIndex', - type: 'SomeType' - } - } -} - -let search = Contexture({ - schemas, - providers: { - elasticsearch: provider({ - getClient, - request: { - headers: { - 'custom-header-app-name': 'my-app-sent-this' - } - }, - types: types() - }) - } -}) -``` - -The code above will provide a working search function `search` that -will transform any given query into a working ElasticSearch query, -which will be sent to the database to retrieve the data. Let's examine -this code in greater detail. - -1. The Dependencies - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-elasticsearch') -let types = require('contexture-elasticsearch/types') -let elasticsearch = require('elasticsearch') -let AgentKeepAlive = require('agentkeepalive') -``` - -The first six lines are about requiring dependencies, the first -dependency being just `contexture`. The second one is our database -provider, `contexture-elasticsearch`. Then, we require the types; even -though you will probably have to write your own types, we have some -pre-defined types available at `contexture-elasticsearch/types`. -Lastly, we require `elasticsearch` and `agentkeepalive`, so we can -actually connect to ElasticSearch and keep it connected even if the -address becomes unreachable for a while. - -Please feel free to dig around these topics by following these links: - -- [Contexture Providers](under-the-hood/contexture-providers/README.md). -- [Types and Type Components](types/README.md). -- [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). -- [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). - -2. The ElasticSearch Client - -```javascript -let elasticClient = null -let getClient = () => { - if (elasticClient) return elasticClient - elasticClient = elasticsearch.Client({ - minSockets: 1, - maxSockets: 20, - keepAlive: true, - createNodeAgent: (connection, config) => - new AgentKeepAlive(connection.makeAgentConfig(config)) - }) - return elasticClient -} -``` - -Now, we define a utility function to connect to ElasticSearch just -once. The idea here is to be able to re-use the same connection -instead of using new clients every time a search is executed. We do -this by declaring a function that will return `elasticClient` if it has -a truthy value, or otherwise instantiate a new elasticsearch client -with the given configuration. The configuration we are sending is -merely an example and shouldn't be used without understanding what is -being expected by the elasticsearch library. More on the following -links: - -- [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). - -3. The Schemas - -```javascript -let schemas = { - elasticsearch: { - index: 'SomeIndex', - type: 'SomeType' - } -} -``` - -As shown above, the next thing we do is to define the schemas. This is -an object which properties are the names of each one of the schemas -that will be available for the contexture DSL. The schemas for the -ElasticSearch provider can specify any or all of the following -properties: - -| Option | Type | Description | Required | -| ------ | ---- | ----------- | -------- | -| `index` | `string` | Which ES index to use when querying | x | -| `type` | `string` | Which ES type to use when querying | | -| `summaryView` | `function` | Used by `results` to return a summary view instead of the whole document, (e.g. for indexes with many fields). Defaults to returning the `hit` property. | | -| `highlight` | `object` | Used by `results` to determine what fields to highlight, and whether or not they are `inline` (copied over inline on to the source) or `additional` (in a list of additional fields that matched) | | -| `forceExclude` | `array` | Used by `results` to extend the exclude fields provided on the search tree. The extension happens only if the results node has a `forceExclude` flag set to true. - -You can read more about these here: - -- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). -- [Schemas section on our Querying docs](../querying/schemas.md). - -4. Our Search Function - -```javascript -let search = Contexture({ - schemas, - providers: { - elasticsearch: provider({ - getClient, - request: { - headers: { - 'custom-header-app-name': 'my-app-sent-this' - } - }, - types: types() - }) - } -}) -``` - -Once we have the schemas set, we can create our `search` function. -For this purpose, we will be calling `Contexture` with just one -object. This object will have the `schemas`, and the `providers`. The -providers will host just one key/value, the one specific for -`elasticsearch`. The provider, which we required at the beginning of -the script, needs to be called with the `getClient` function we just -created and the `types()` that we got from the -`contexture-elasticsearch/types` repository. We also show that you can -customize the request headers by providing an object that includes the -headers keys and values. This `search` function is ready to receive -search trees and write the results back! - -You can read more about these topics in the following links: - -- [Contexture Core's Default Export](../under-the-hood/contexture-core.md#default-export). -- [Contexture Providers in detail](under-the-hood/contexture-providers/README.md). - -This example and many other important details about -`contexture-elasticsearch` are accessible in the following links: - -- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). - -### Connecting to MongoDB - -The followng code example shows how to connect to MongoDB: - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-mongo') -let types = require('contexture-mongo/types') -let MongoClient = require('mongodb').MongoClient - -let schemas = { - yourCustomSchemaName: { - mongo: { - collection: 'SomeCollection' - } - } -} - -let search = null - -MongoClient.connect('mongodb://localhost:27017', function(err, client) { - search = Contexture({ - schemas, - providers: { - mongo: provider({ - getClient: () => client, - types: types() - }) - } - }) -}) -``` - -The code above will provide a working search function `search` that -will transform any given query into a working MongoDB query, -which will be sent to the database to retrieve the data. Let's examine -this code in greater detail. - -1. The Dependencies - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-mongo') -let types = require('contexture-mongo/types') -let MongoClient = require('mongodb').MongoClient -``` - -The first lines are about requiring dependencies, the first -dependency being just `contexture`. The second one is our database -provider, `contexture-mongo`. Then, we require the types; even -though you will probably have to write your own types, we have some -pre-defined types available at `contexture-mongo/types`. -Lastly, we require `mongodb` to get the much needed `MongoClient`, so -we can actually connect to the database. - -Please feel free to dig around these topics by following these links: - -- [Contexture Providers](under-the-hood/contexture-providers/README.md). -- [Types and Type Components](types/README.md). -- [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). - -2. The Schemas - -```javascript -let schemas = { - yourCustomSchemaName: { - mongo: { - collection: 'SomeCollection' - } - } -} -``` - -As shown above, the next thing we do is to define the schemas. This is -an object which properties are the names of each one of the schemas -that will be available for the contexture DSL. The schemas for the -MongoDB provider can specify any or all of the following -properties: - -| Option | Type | Description | Required | -| ------ | ---- | ----------- | -------- | -| `collection` | `string` | The MongoDB collection that will be used to run the queries | x | - -You can read more about these here: - -- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). -- [Schemas section on our Querying docs](../querying/schemas.md). - -3. The MongoDB Client & Our Search Function - -```javascript -let search = null - -MongoClient.connect('mongodb://localhost:27017', function(err, client) { - search = Contexture({ - schemas, - providers: { - mongo: provider({ - getClient: () => client, - types: types() - }) - } - }) -}) -``` - -Next we will need to connect to MongoDB. In the previous code, we -provide an example in which we connect to `mongodb://localhost:27017`, -and using a callback to the `connect` method, we are able to obtain -the database client that we need. Once we have this client, we can -actually create our `search` function. For this purpose, we will -be calling `Contexture` with just one object. This object will have -the `schemas`, and the `providers`. The providers will host just one -key/value, the one specific for `mongo`. The provider, which -we required at the beginning of the script, needs to be called with -a `getClient` function that will just return the database client, and -the `types()` that we got from the `contexture-mongo/types` repository. -With these steps completed, we end up with a function that is ready to -receive search trees and write the results back! - -You can read more about these topics in the following links: - -- [Contexture Core's Default Export](../under-the-hood/contexture-core.md#default-export). -- [Contexture Providers in detail](under-the-hood/contexture-providers/README.md). -- [Connecting to MongoDB using the native MongoDB driver for NodeJS](http://mongodb.github.io/node-mongodb-native/api-articles/nodekoarticle1.html#getting-that-connection-to-the-database). - -This example and many other important details about -`contexture-mongo` are accessible in the following links: - -- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). - -[↪ Next: Connecting to Other Databases](connecting-other-databases.md) diff --git a/getting-started/discovering-the-database.md b/getting-started/discovering-the-database.md deleted file mode 100644 index 0cf2629..0000000 --- a/getting-started/discovering-the-database.md +++ /dev/null @@ -1,121 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Your First Filter](your-first-filter.md) - -## Discovering The Database - -One of the great things about Contexture is that it can be used to -discover the database. In this page, we'll see how to write a filter -that not only allows the user to refine their search, but that also -shows information about our data that might not be obvious by looking -at the results directly. - -### The Facet Filter - -The `facet` type is just like any other type in the tree. It requires -a unique key and some other properties. In contrast to previously seen -types, such as `text` and `number`, facet doesn't require specific -values, but instead it asks for two properties: `field`, and -optionally `size`. This type is incredibly useful because besides -filtering the results based on the `value` (or `values`) the user -might have chosen, it retrieves from the database a list of available -values! The `facet` type can be very well represented as a list of -checkboxes, ready for users to pick for one or more values. Let's see -how we can use it. - -### Adding the Facet Filter to the Tree - -To add the `facet` filter to the tree, we simply add it to the -structure we already had. For example: - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'numberrange', - type: 'number', - field: 'age', - min: 0, - max: 100 - }, { - key: 'cityfacet', - type: 'facet', - field: 'city', - value: 'Orlando', // Optional - // We could have instead `values` with ['Orlando', 'Miami'] - }, { - key: 'results', - type: 'results' - }] -} -``` - -Once we have it in the tree, we can simply create another component to -allow users to set a value to this type. In this case, however, we -will be retrieving the available values from a field that is -automatically inserted to this part of the tree. Let's make sure we -have the data before we render the components. - -### Fetching the Available Options - -Having the facet node already added to the tree, it's only matter of -running a first search before we render the components to actually get -the possible results from the `facet` type. We can achieve this by -calling to `contexture-client`'s `refresh` function: - -```javascript -// This would go after: -// let contextureSearchTree = Contexture(searchTree) -contextureSearchTree.refresh(['root']) -``` - -Now, the available options will be ready to be used at: -`contextureSearchTree.getNode(['root', 'cityfacet']).context.options`. -So we can write our facet component this way: - -```javascript -let toggleValue = (array, value) => { - if (array.indexOf(value) > -1) { - let copy = array.slice() // Making sure it's not a MobX Array - copy.splice(array.indexOf(value), 1) // removing value from the array - array.replace(copy) // MobX Arrays have this method to replace the array's inner values - } else { - array.push(value) - } -} -let RangeComponent = observer(({ tree }) => ( -
- Select One or More: - {tree.getNode(['root', 'cityfacet']).context.options.map(option => ( -
- -1} - onChange={() => toggleValue(tree.getNode(['root', 'cityfacet']).values, option.value)} - /> - -
- ))} -
-)) -``` - -Including this component in our search interface will show the current -available cities for the search results we have so far, and will allow -users to filter the search even more by letting them pick any (or many) -of the available cities. - -You can read more about our available types here: - -- [Types and Type Components](../types/README.md) - -[↪ Next: IMDB Index](imdb-example.md) diff --git a/getting-started/first-script.md b/getting-started/first-script.md deleted file mode 100644 index 8f73de5..0000000 --- a/getting-started/first-script.md +++ /dev/null @@ -1,202 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Project Setup](setup.md) - -## Your First Contexture Script - -With everything installed, let's see a simple script that runs a -simple one-time search: - -```javascript -let contexture = require('contexture') - -let schemas = { - collectionNameSchema: { - mongo: { - collection: 'collectionName' - } - } -} - -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'results', - type: 'results' - }] -} - -let result = await contexture({ - schemas, - providers: { - mongo: require('contexture-mongo')({ - types: require('contexture-mongo/types')(), - getClient: () => ({ - // Fake mongo client. - // For this example we only care about - // collection().aggregate([...]).toArray() being a promise. - collection: name => ({ - aggregate: aggregations => ({ - toArray: async () => ['Unrealistic result example'] - }) - }) - }) - }) - }, -}, searchTree) - -console.log(result.children[1].context.response.results) - -// Explore it yourself! -result -``` - -You can also try this same code in Runkit here: - - -## What it does - -1. Requiring Contexture - -```javascript -let contexture = require('contexture') -``` -We start by requiring `contexture`. There's nothing much else to see -here. - -![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) - -2. Writing the Schemas - -```javascript -let schemas = { - collectionNameSchema: { - mongo: { - collection: 'collectionName' - } - } -} -``` -Immediatly afterwards, we define our schemas. For this specific -example, we are going to emulate doing a search on a single collection -of a Mongo database. We define `collectionName` as the name of this -arbitrary collection. The `schemas` object ends up containing a single -schema with a key being `collectionNameSchema`, which is going to be -supported by a single provider `mongo`, from which we'll be looking -for the `collectionName` collection. We'll learn more about the -schemas later on, in [Querying → Schemas](../querying/schemas.md). - -3. Writing the Contexture DSL - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'results', - type: 'results' - }] -} -``` -Our next step is to define a simple search query using Contexture DSL. -This search query will have a mandatory root group, which among other -features indicates which schema we will be running this query on -(`collectionNameSchema`). This node will have two children. The first -children is going to be our search query. This query will be a `text` -type query. We're indicating that we want to run a plain text search -that will try to match any record which `name` contains the word -`value`. The next node has the `results` type. This is where the -results will be written once the search runs. - -You can read more about these topics in the following links: - -- [Querying](../querying/README.md). -- [Types and Type Components](../types/README.md). -- [Mongo Example Types](../types/mongo-example-types.md). - -3. Getting Results - -```javascript -let result = await contexture({ - schemas, - providers: { - mongo: require('contexture-mongo')({ - types: require('contexture-mongo/types')(), - getClient: () => ({ - // Fake mongo client. - // For this example we only care about - // collection().aggregate([...]).toArray() being a promise. - collection: name => ({ - aggregate: aggregations => ({ - toArray: async () => ['Unrealistic result example'] - }) - }) - }) - }) - }, -}, searchTree) -``` - -The next thing we do is to actually run a one-time search. In this -case, we `await` for `contexture`, passing along some very important -parameters. First, we pass the schemas we previously defined. Then, we -pass a providers object, which has the provider for `mongo`. This -`mongo` key will be matched against the `mongo` key that we defined in -the schemas, so you could change this property name to something -entirely different. When we send the mongo provider, we assign the -result of the initialization of `contexture-mongo`, where we send the -`contxture-mongo/types` (which **needs to be called as a function once -required**) and a `getClient` function. In a real schenario, you would -just send the object that results of `require('mongodb')`. However, to -provide an exceutable example in the browser, we've made a very small -Mock of MongoDB where we will return a same object for any collection -call, which will only allow fake aggregations, which will have a -promise `toArray` function that will return our `Unrealistic result -example`. - -Keep in mind that since `contexture` returns a promise, you can change -`await contexture({ /* .. */ })` to `contexture({ /* .. */ }).then()`, -but you'll need to move the code that we have after the await call -into the function that is passed on the `.then()` call. - -You can read more about these concepts here: -- [Querying](../querying/README.md). -- [Contexture Providers](../under-the-hood/contexture-providers/README.md). -- Contexture Core's [Default Export](../under-the-hood/contexture-core.md#default-export). - -4. Exploring the Results - -```javascript -console.log(result.children[1].context.response.results) - -// Explore it yourself! -result -``` - -Finally, we can use our search result! The output of the search will -be added to a copy of the original search query. The location of this -result will be in the children of the `root` node that has the -`results` type, within a `context` object, within a `response` object, -on a `results` property. This whole object is going to be exposed in -the _runkit_ example for you to play with it. For more information, -you can dive in here: - -- [Contexture DSL](../querying/contexture-dsl.md). -- [Design Principles](../under-the-hood/design-principles.md). -- [Contexture Core](../under-the-hood/contexture-core.md) - -[↪ Next: Connecting to ElasticSearch & MongoDB](connecting.md) diff --git a/getting-started/imdb-example.md b/getting-started/imdb-example.md deleted file mode 100644 index bf64757..0000000 --- a/getting-started/imdb-example.md +++ /dev/null @@ -1,73 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Your First Filter](your-first-filter.md) - -## IMDB Index - -In our Contexture React repository, we've created a Storybook live -example that uses Contexture (with no server whatsoever, everything in -the client) to query and discover a public ElasticSearch index with -data similar to the data that IMDB might contain. - -- [Contexture React Repository](https://github.com/smartprocure/contexture-react). -- More information about [Storybook](https://github.com/storybooks/storybook). -- [The IMDB Live Index Explorer with Contexture](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). -- [The IMDB Live Index Explorer Source Code](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). - -Let's see how we can use this tool. - -### Picking a Field - -The tool initially shows you a button named `Pick a Field`: - -![](https://i.imgur.com/CWz4Moy.png) - -This button will allow you to select any field on which you might want -to run a search. If you click it, you'll see the following fields: - -![](https://i.imgur.com/RR6vMP1.png) - -If we pick `Actors`, for example, the search will trigger and results -will appear: - -![](https://i.imgur.com/kep1LIo.png) - -Now, let's click on `Select Type` and click on `Facet`. A moment -afterwards, we will get a list of checkboxes with the most common -actors among the results: - -![](https://i.imgur.com/2Gq5FEh.png) - -We can see that the actor that has made the most movies (among the -ones idexed) is Naveen Andrews. We can also see the number of movies -this and other actors have made, and the total actors in the database -(of which we can only see 10 in this screenshot). Under this list of -actors, you'll see a text saying `View More`. If you click it, the -next 10 actors will appear (in order). - -At the bottom of the page, you'll see a button labeled `Add Filter`. -Clicking it will add another interactive component to the website, -which will say `Click to add AND`. - -![](https://i.imgur.com/sRHN04n.png) - -Clicking the `Click to add AND` button will show us again a `Pick a -Field` button: - -![](https://i.imgur.com/oA4LOEK.png) - -So we can start again. Let's say we pick the `Genre` field, and we -click for the `Facet` type again. A bit later, the list of ordered -genres will appear: - -![](https://i.imgur.com/nfbVlQ2.png) - -And that's it! We're discovering the database with a very simple and -unpolished interface. By this point you might be curious on what -components we're using to do all this magic. There's really no trick, -just follow us through the tutorial. You can also skip some steps if -you're particularly interested in: - -- [Contexture React Repository](https://github.com/smartprocure/contexture-react). -- [Available React Components for Types](../types/react-components.md). - -[↪ Next: Querying](../querying/README.md) diff --git a/getting-started/setup.md b/getting-started/setup.md deleted file mode 100644 index dfb6054..0000000 --- a/getting-started/setup.md +++ /dev/null @@ -1,74 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Getting Started](README.md) - -# Project Setup - -The whole Contexture Framework is available through NPM. It doesn't -have any extra dependency besides Node 9 and some NPM repositories. In -this quick guide, we'll achieve the following goals: - -- [Install Node 9 and NPM](./setup.md#installing-node-9-and-npm) -- [Install Contexture](./setup.md#installing-contexture) -- [Install Contexture Client](./setup.md#installing-contexture-client) -- [Install Contexture ElasticSearch](./setup.md#installing-contexture-elasticsearch) -- [Install Contexture Mongo](./setup.md#installing-contexture-mongo) -- [Install Contexture React](./setup.md#installing-contexture-react) - -**Note:** as you progress through our documentation, you'll discover -that in some cases, you will need only one or two of these -repositories. This page just provides the fastest way to start that we -couldcome up with. - -## Installing Node 9 and NPM - -NodeJS 9 and NPM can be installed through [the list of previous -releases of NodeJS](https://nodejs.org/en/download/releases/). You -might get it working with Node 10 (if so, let us know). We haven't -fully upgraded to Node 10 yet, so until then, we encourage you to at -least have a working version of Node 9 in hand. - -An easy way to move from one version to another is with -[nvm](https://github.com/creationix/nvm). Here's a command you can run -to install nvm: - - curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash - -Please follow [nvm's README](https://github.com/creationix/nvm/blob/master/README.md) for more information. - -## Installing Contexture - -Once you have NodeJS and NPM installed, you'll need either a new -folder for your new project with Contexture, or to go to an existing -project, then run: - - npm install --save contexture - -## Installing Contexture Client - -To install the `contexture-client` you can run the following -command in your project's root folder: - - npm install --save contexture-client - -## Installing Contexture ElasticSearch - -To install the `contexture-elasticsearch` you can also run the -following command in your project's root folder: - - npm install --save contexture-elasticsearch - -## Installing Contexture Mongo - -To install the `contexture-mongo` you can also run the following -command in your project's root folder: - - npm install --save contexture-mongo - -## Installing Contexture React - -To install the `contexture-react` you can also run the following -command in your project's root folder: - - npm install --save contexture-react - -[↪ Next: A First Contexture Script](first-script.md) diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md deleted file mode 100644 index cb7f93b..0000000 --- a/getting-started/simple-search-box.md +++ /dev/null @@ -1,240 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Connecting to Other Databases](connecting-other-databases.md) - -## Simple Search Box - -Building a simple search box consists of a single input field tied to -any of the fields that any record might have in the specified database -or index. To be able to make this text input tied to the search -structure, we will need to bring `contexture-client` in. With that in -mind, and the knowledge we already have of contexture, we can define -the following tasks: - -1. Create a New Contexture Search Function. -2. Create a Web Server With a Search Endpoint -3. Write a Search Tree. -4. Make the Search Tree Aware of Contexture. -5. Write a Text Input. - -Let's dive in. - -### 1. Creating a New Conteture Search Function - -Just as how we saw in the previous pages, creating a new Contexture -search function is about setting up the `contexture` package's default -export with `schemas` and `providers`. In this case, we'll use the -contexture-mongo approach in the following file (let's call it -`search.js`): - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-mongo') -let types = require('contexture-mongo/types') -let MongoClient = require('mongodb').MongoClient - -let schemas = { - collectionNameSchema: { - mongo: { - collection: 'collectionName' - } - } -} - -module.exports = {} - -MongoClient.connect('mongodb://localhost:27017', function(err, client) { - module.exports.search = Contexture({ - schemas, - providers: { - mongo: provider({ - getClient: () => client, - types: types() - }) - } - }) -}) -``` - -Note that we're now exporting the search function with -`module.exports.search = Contexture(...`. You can also note that we -specified the schema's name to be `collectionNameSchema`, and that any -search using this schema will be using MongoDb's `collectionName` -collection. - -### 2. Create a Web Server With a Search Endpoint - -If we want to separate the direct access to the database from the -client, the previous code should live in the server. Our next step is -to expose a `/search` endpoint so our future client can reach this -function. In this section, we'll write a simple web server with -`express` to satisfy our needs. - -#### 2.1. Installing the Dependencies - -You'll need to install `express` and `body-parser` at the root of your -project, as follows: - - npm install --save express body-parser - -#### 2.2. Writing the Web Server - -Once you have the dependencies installed, you can set up the server -with the following code: - -```javascript -let express = require('express') -let bodyParser = require('body-parser') -let search = require('./search') -let app = express() - -// create application/json parser -lete jsonParser = bodyParser.json() - -app.post('/search', jsonParser, (req, res) => { - if (!req.body || !req.body.search) return res.sendStatus(400) - search(req.body.search).then((err, result) => { - if (err) return res.send(401, err) - res.send(200, result) - }) -}) - -app.listen(3000) -``` - -You can read more about these topics in the following links: - -- [Express Documentation](https://expressjs.com/en/api.html). -- [body-parser repository](https://github.com/expressjs/body-parser). - -### 3. Writing a Search Tree - -Having the DSL processor available through a web server endpoint, we -can follow up with the structure of the search interface itself. We'll -conceptualize this by writing the Contexture DSL itself. - -Let's use the same `sarchTree` that we used in [our frist -script](first-script.md): - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'results', - type: 'results' - }] -} -``` - -Keep in mind that we'll be using `collectionNameSchema` since we -already defined a schema with that name on the server's `search.js` -file. - -You can read more about writing Contexture DSL queries in our -[Querying](../querying/README.md) documentation. - -### 4. Make the Search Tree Aware of Contexture - -Having a search tree, we will need `contexture-client` to make it -smart enough for the user interface. Let's make sure we have it -available in our project with: - - npm install --save contexture-client - -We will also use MobX to make it easier to notify our component that -the state has changed. For that purpose, we will need to install -`mobx` using NPM: - - npm install --save mobx - -Since we're heavy users of MobX, `contexture-client` already -provides an adapter that we can use out of the box. Knowing this, -let's prepare our `contexture-client` to work well with -`mobx`, as follows: - - -```javascript -let ContextureClient = require('contexture-client') -let ContextureMobx = require('contexture-react/dist/utils/contexture-mobx') - -let types = ContextureClient.exampleTypes -let service = async search => ({ - data: await postData('/sarch', { search }) -}) - -let Contexture = ContextureMobx({ - types, - service, -}) -``` - -Note that our service function will be the one responsible for sending -the `searchTree` we previously defined to the DSL processor, we can -wrap the search tree into a smart object that will later react to both -the user input, and the search results. - -```javascript -let contextureSearchTree = Contexture(searchTree) -``` - -You can read more about these topics here: - -- [MobX](https://mobx.js.org/). -- [MobX Observers](https://mobx.js.org/refguide/observer-component.html). -- [Managing the Contexture Search Tree State with MobX](../managing-state/mobx.md). -- [Managing the Contexture Search Tree State with MobX](../managing-state/mobx.md). -- [Introduction to the Contexture Client for Easy Querying](../querying/interactive-queries/contexture-client.md). -- [Introduction to the Contexture Client for Easy Querying](../querying/interactive-queries/contexture-client.md). -- [Contexture Client in Detail](../under-the-hood/contexture-client.md). - -### 5. Writing a Text Input - -Having the search tree ready allows us to write a `mobx` observer -component that will receive the tree and react to the result changes -immediately. Here's an example: - -``` -let { observer } = require('mobx') -let SearchQuery = observer(({ tree }) => - { - tree.getNode(['root', 'namequery']).value = e.target.value - }} - /> - -let SearchResults = observer(({ tree }) => ( -
- {JSON.stringify(tree.getNode(['root', 'results']).context.response.results)} -
-)) -``` - -Which we would render this way: - -```javascript - - -``` - -This will generate a text input that will trigger a new search every -time the contents of the input change. This new search will get the -results and show the results in a JSON form (because of the -`JSON.stringify` part). Ideally, you will not render them in a JSON -form, but render them using a list component or table. - -More information here: - -- [Interactive Queries](../interactive-queries/README.md). -- [Available React Components for our Available Types](../types/react-components.md). -- [Managing State](../managing-state/README.md). -- [Recommendations](../recommendations/README.md). - -[↪ Next: Your First Filter](your-first-filter.md) diff --git a/getting-started/your-first-filter.md b/getting-started/your-first-filter.md deleted file mode 100644 index 88d0278..0000000 --- a/getting-started/your-first-filter.md +++ /dev/null @@ -1,74 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Simple Search Box](simple-search-box.md) - -## Your First Filter - -In the previous page, we wrote a simple search user interface where a -single input would retrieve results. Now, we will add a simple filter -that will allow us to get results within a given range. - -### Adding the Filter to the Tree - -The way we will add the filter is by adding a node to the tree. This -node is going to have a type of `number`, which asks for a `field`, a -`min` and a `max` values. - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'numberrange', - type: 'number', - field: 'age', - min: 0, - max: 100 - }, { - key: 'results', - type: 'results' - }] -} -``` - -With the `numberrange` node added, we can create another component -with two inputs that will allow us to filter results with the range -that the user specifies: - -```javascript -let RangeComponent = observer(({ tree }) => ( -
- Minimum: - { - tree.getNode(['root', 'numberrange']).min = e.target.min - }} - /> -
- Maximum: - { - tree.getNode(['root', 'numberrange']).max = e.target.max - }} - /> -
-)) -``` - -And that's it! Rendering this component will make our search aware of -any change the user might desire on the minimum and maximum ages they -might want to use to filter the available results. - -You can read more about our available types here: - -- [Types and Type Components](../types/README.md) - -[↪ Next: Discovering the Database](discovering-the-database.md) diff --git a/managing-state/README.md b/managing-state/README.md deleted file mode 100644 index 8b8ceb8..0000000 --- a/managing-state/README.md +++ /dev/null @@ -1 +0,0 @@ -# Managing State diff --git a/other-components/README.md b/other-components/README.md deleted file mode 100644 index 778d7fb..0000000 --- a/other-components/README.md +++ /dev/null @@ -1,6 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - -# Other Components - -- [Generally Useful Components](general.md). -- [Layout Components](layout.md). diff --git a/other-components/general.md b/other-components/general.md deleted file mode 100644 index 4b0c3ec..0000000 --- a/other-components/general.md +++ /dev/null @@ -1,85 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Other Components](README.md) - -## Generally Useful Components - -### ContextureProvider - -This component is a magic tool to make Contexture search interfaces without -having to write the tree separated. With this component, you can pass the tree -directly as a plain object, and any children you pass will receive the `tree` -property directly. - -This component receives: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `types` | Object | Yes | Your client-side types, such as the default types of the Contexture Client. | -| `service` | Object | Yes | The search function that sends the DSL to the initialized Contexture Core. | -| `nodeKey` | Object | Yes | Key for the root node. Defaults with `root`. | -| `...props` | Any other property (children excluded) | Yes | Any other property that you might want to send to the initialization of the Contexture Client. | - -**Note:** Any of the Contexture React Example Types components automatically -add the nodes to the tree if the referenced node is not present. _This is -an experimental feature_. - -Here's how you write your component: -```javascript -let ContextureProvider = require('contexture-react/dist/ContextureProvider') -let ContextureClient = require('contexture-client') -let types = ContextureClient.exampleTypes -let service = async search => ({ - data: await postData('/sarch', { search }) -}) -// ... -// Later, on your render function, or where you put your components: - - - -``` - -- [Source code of the ContextureProvider component](https://github.com/smartprocure/contexture-react/blob/master/src/ContextureProvider.js). - -### FilterList - -A component that tries to automatically render the specific type components of -the children of a node. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `node` | Object | Yes | Node of the Contexture Tree where the children that want to be rendered are. | -| `exampleTypes` | Object | No | Object which a key and a component per type. Defaults in the available example types components in contexture-elasticsearch. | -| `fields` | Object | Yes | Object which a key and an object with at least a `label` string property, used to indicate the label of each one of the filters. | - -Here's how you write your component: -```javascript -let FilterList = require('contexture-react/dist/FilterList') -let ContextureClient = require('contexture-client') -let tree = ContextureClient({ - key: 'root', - type: 'group', - schema: 'mySchema', - children: [{ - key: 'query', - type: 'query', - field: 'myFieldName', - query: 'Something' - }] -}) -// ... -// Later, on your render function, or where you put your components: - -``` - -- [Source code of the FilterList component](https://github.com/smartprocure/contexture-react/blob/master/src/FilterList.js). - -Up next, components only useful for helping on building the layout. - -[↪ Next: Layout Components](layout.md) diff --git a/other-components/layout.md b/other-components/layout.md deleted file mode 100644 index 9fb22b1..0000000 --- a/other-components/layout.md +++ /dev/null @@ -1,120 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Generally Useful Components](general.md) - -## Layout Components - -### Awaiter - -Renders a loading indicator until a Promise is resolved. It will ender -an exception (currently just `Ooops...`) if the Promise fails. if the -Promise passes, the children are rendered. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `promise` | Promise | Yes | Minumum number for the range. | - -Here's how you write your component: -```javascript -let Awaiter = require('contexture-react/dist/layout/Awaiter') -let promiseMaker = () => new Promise((resolve, reject)) /* ... */ resolve()) -// ... -// Later, on your render function, or where you put your components: - -
My Crazy Children
- -``` - -- [Source code of the Awaiter component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/Awaiter.js). - -### BarChart - -Allows you to build your own bar charts (besides just relying on -the DateHistogram component). - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `valueField` | String | `''` | The field of each record where the value used for the height of each barnis located. | -| `categoryField` | String | `''` | The field of each record where the label of each bar is located. | -| `data` | Array (of Objects with both the `valueField` and the `categoryField`) | `[]` | The data that is going to be used to build the chart. | -| `height` | Number | `100` | Specifies the max height of the whole chart. | -| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | -| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | -| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | - -Here's how you write your component: -```javascript -let BarChart = require('contexture-react/dist/layout/BarChart') -// ... -// Later, on your render function, or where you put your components: - -``` - -- [Source code of the BarChart component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/BarChart.js). - -### SpacedList - -Wraps every children in a div with a given style. Useful for (as the -name portrays) making a spaced list. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `style` | Object | No | The style that will be applied to each div. Defaults in `{ marginBottom: '25px' }`. | - -Here's how you write your component: -```javascript -let SpacedList = require('contexture-react/dist/layout/SpacedList') -// ... -// Later, on your render function, or where you put your components: - -

Hi

- - -``` - -- [Source code of the SpacedList component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/SpacedList.js). - -### TextHighlight - -Used to highlight content within a text basef on a pattern. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `pattern` | String | No | RegExp pattern used to find matching content. | -| `text` | String | Yes | Text used as the target of the matches. | -| `Wrap` | Component | No | Component used to wrap each one of the -matched. Defaults to `i`. | - -Here's how you write your component: -```javascript -let TextHighlight = require('contexture-react/dist/layout/TextHighlight') -// ... -// Later, on your render function, or where you put your components: - -``` - -- [Source code of the TextHighlight component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/TextHighlight.js). - -Up next, our recommendations on state management. - -[↪ Next: Managing State](../managing-state/README.md) diff --git a/querying/README.md b/querying/README.md deleted file mode 100644 index 51d9b0c..0000000 --- a/querying/README.md +++ /dev/null @@ -1,11 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - -# Querying - -Table of Contents: -- [Contexture DSL](contexture-dsl.md) -- [Interactive Queries](interactive-queries/README.md) - - [Contexture Client](interactive-queries/contexture-client.md) - - [Initializing the Contexture Client](interactive-queries/contexture-client.md#initializing-the-contexture-client) - - [Context Tree](interactive-queries/contexture-client.md#context-tree) - - [Introduction to Reactors](interactive-queries/reactors.md) diff --git a/querying/contexture-dsl.md b/querying/contexture-dsl.md deleted file mode 100644 index 92231f2..0000000 --- a/querying/contexture-dsl.md +++ /dev/null @@ -1,58 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Querying](README.md) - -## Contexture DSL - -The Contexture DSL is a JavaScript object structure, or JSON -structure, that is composed of nested nodes that are equal at all -levels. Each node has a `key`, a `type` and many other optional -properties. The first node is called the root and is a node of type -`group`. Any node of type `group` can have many children. Each -children can be a node of any type. If any children is another group, -this children will probably have one or more other children nodes of -any type, and so on. - -Let's begin talking about the root. - -### The Root - -The root of a Contexture Tree is a node of type `group`. Group types -are required to have a `key`, the `type: "group"` property, and an -array of children `children: [/* nodes */]`. Root groups also need an -extra property: then ame of the schema. In summary, this is how a root -node should look: - -```javascript -let searchTree = { - key: 'myRootNode', - type: 'group', - schema: 'mySchemaName', - children: [ - // Other nodes - ] -} -``` - -### The Children - -Each children will be an individual node. Each node will have at least -a unique `key` (unique per tree, but they can appear again in other -trees), and a `type`. Some types might require more properties. You -can learn more about our types at: - -- [Types and Type Components](types/README.md) - -That's really all that it is for our DSL. The magic happens in the -types and the providers. As long as your types are defined properly, -and your providers build the correct queries and write them back in -the tree, Contexture will work and you'll be able to use the -Contexture DSL to build search interfaces of any complexity. - -**Note:** Many of the types have some common properties, like `field` -or `values`. The only reason these properties are common is because -they've made sense to be common for each one of our specific types, -not because they have something to share between types. Each type has -it's own properties and rules, so you should treat them as independent -structures. - -[↪ Next: Interactive Queries](interactive-queries/README.md) diff --git a/querying/interactive-queries/README.md b/querying/interactive-queries/README.md deleted file mode 100644 index e94fddf..0000000 --- a/querying/interactive-queries/README.md +++ /dev/null @@ -1,13 +0,0 @@ -[↩ Parent: Querying](../README.md) - -# Interactive Queries - -Table of Contents: -- [Contexture Client](contexture-client.md) - - [Initializing the Contexture Client](contexture-client.md#initializing-the-contexture-client) - - [Contexture Client Types](contexture-client.md#contexture-client-types) - - [Contexture Client Service](contexture-client.md#contexture-client-service) - - [Context Tree](contexture-client.md#context-tree) - - [Tree and getNode](contexture-client.md#tree-and-getnode) - - [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) -- [Introduction to Reactors](reactors.md) diff --git a/querying/interactive-queries/contexture-client.md b/querying/interactive-queries/contexture-client.md deleted file mode 100644 index a0e0fd1..0000000 --- a/querying/interactive-queries/contexture-client.md +++ /dev/null @@ -1,179 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Interactive Queries](README.md) - -## Contexture Client - -Having a Contexture DSL by itself won't work since it's only a -language and not an interactive or working program in any sense. We -need something to automatically send this query to our DSL processor -(the main `contexture` repository). For this purpose, we provide the -`contexture-client`. - -The Contexture Client is responsible for keeping track of the changes -that happen on the Contexture DSL, either by the user or by the -Contexture core. The nodes in the Contexture DSL host values used for -filtering the results, but also hold other properties, such as the -search results, and a `_meta` object that you can use for debugging -purposes. In this page, we will sumarize how the Contexture Client -works with the Contexture DSL to provide real-time interactive -searches. - -### Initializing the Contexture Client - -The Contexture Client has a default export that needs to be called -first with some configuration properties before being able to process -any search. This is how you would normally initialize it: - -```javascript -let ContextureClient = require('contexture-client') -let Contexture = ContextureClient({ - service, - types, -}) -``` - -When Contexture Client determines the search query is ready to be -executed, it will call the `service` function provided with the whole -Contexture DSL. It expects to receive another full DSL that it parses -to then change the nodes again. As previously mentioned, an example -`service` would be: - -```javascript -let service = async search => ({ - data: await postData('/sarch', { search }) -}) -``` - -The `types` property is an object where each key is a type's name. -These type definitions are different than the ones from the servers -and allow our Contexture Client to do three things: - -- To know how to validate a node. -- To know what happens to a specific node (and it's suroundings) when - it changes. -- To know how to complemente any missing value on a specific node - (default values, so you can omit properties when you write each of - the nodes). - -Each one of the types will have the following properties: - -| Property Name | Type | Description | -| --- | --- | --- | -| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | -| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | -| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | - -For more information, check the following links: - -- [How to make your own types (client and provider)](../../types/diy-types.md). -- [Specifically on how to make client types](../../types/diy-types.md#how-to-wite-a-client-type). - -### Context Tree - -Once you have Contexture Client initialized, we will be able to pass -the Contexture DSL into it. This will generate something we call the -`Context Tree`, which is an object with the following properties: - -| Name | Signature | Description | -| ---- | --------- | ----------- | -| add | `async (path, newNode) -> await searchCompleted` | Adds a node to the tree as a child of the specified path. You can await this for when updates settle and relevant searches are completed. | -| remove | `async path -> await searchCompleted` | Removes a node at the specified path. You can await this for when updates settle and relevant searches are completed. | -| mutate | `async (path, deltas) -> await searchCompleted` | Mutates the node at the given path with the new values. You can await this for when updates settle and relevant searches are completed. | -| getNode | `[path] -> node` | Lookup a node by a path (array of keys). | -| tree | tree | A reference to the internal tree. If you mutate this, you should dispatch an appropriate event. | - -You can read more about the instantiated client here: [contexture-client Run Time](https://github.com/smartprocure/contexture-client#run-time). - -You can also read more about the Contexture Client in our detailed -documentation: [Under the Hood: Contexture Client](../../under-the-hood/contexture-client.md). - -### Tree and getNode - -Context Trees have two properties for nagivating through the -Contexture DSL: `tree` and `getNode`. The `tree` property will have -the DSL as it was received, plus the default properties set by each -one of the client types, and some state properties. On the other hand, -`getNode` is a quick way to access the tree by sending only the keys -of each node. For example, let's say we have the following DSL: - -```javascript -let searchTree = ContextureClient({ - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }] -}) -``` - -We are able to access the `namequery` node by simply calling: - -```javascript -let nameQueryNode = searchTree.getNode(['root', 'namequery']) -``` - -### Mutate, Add and Remove - -Once you have a Context Tree ready, you will trigger searches -automatically by invoking one of the following functions: `add`, -`remove`, `mutate`. More specifically: - -- If you want to run a search after changing some of the properties of - a specific node, you would call `mutate`. -- If you want to add a children to any given `group`, even the root - group, you would call `add`. -- If you want to remove a children to any given `group`, even the root - group, you would call `remove`. - -Let's see some examples: - -1. Mutate - -Mutate allows us to change properties on nodes and trigger search -afterwards. Here is an example: - -```javascript -await searchTree.mutate(['root', 'namequery'], { - value: 'new value' -}) -``` - -This will change the tree and produce a new set of results. If you're -wondering how to keep track of these changes, the simplest way to do -it is by using our MobX adapter, as shown in our [simple search -box](../../getting-started/simple-search-box.md) tutorial, or in -greater detail in our [Managing State](managing-state/README.md) docs. - -2. Add - -Having the previous search tree, we can add a children by doing: - -```javascript -await searchTree.add(['root'], { - key: 'results', - type: 'results' -}) -``` - -3. Remove - -We can remove the `results` node we just added by doing: - -```javascript -await searchTree.add(['root', 'results']) -``` - -Calling `mutate`, `add` or `remove` will trigger events not only for -the node that these functions are targetting, but also for nearby -nodes, depending on the types. - -Up next, we'll dig a bit into what are the client side reactors (the -rules that Contexture Client follows to know what other nodes are -relevant for each update). - -[↪ Next: Reactors](reactors.md) diff --git a/querying/interactive-queries/reactors.md b/querying/interactive-queries/reactors.md deleted file mode 100644 index cd8b84c..0000000 --- a/querying/interactive-queries/reactors.md +++ /dev/null @@ -1,44 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Contexture Client](contexture-client.md) - -## Reactors - -When any node changes, depending on the type, it might be reasonable -to re-trigger a search call for other nodes. We call this process the -selection of `reactors`, where the possible reactors are only three: -`self`, `others` and `all`. - -Reactors should be specified in the Client Types, where each type will -have a specific reactor for each one of it's properties. For example -(taken from our client side [example -types](https://github.com/smartprocure/contexture-client/blob/master/src/exampleTypes.js)): - -```javascript -facet: { - reactors: { - values: 'others', - mode: 'others', - size: 'self', - optionsFilter: 'self', - sort: 'self', - } -} -``` - -The client side type defined above will be effective for any node with -type `facet`, where the properties `values` and `mode` will affect -only all the other nodes (and not itself), and the properties `size`, -`optionsFilter` and `sort` will affect only the specific `facet` node -and no other node. - -The one remaining reactor that isn't covered by that example is the -`all` reactor. The difference between `others` and `all` is that -`others` excludes the node where the change is happening, and `all` -includes all other nodes and the node where the change is happening -(effectively combining `self` and `others`). - -You can also read more about reactors in the Contexture Client in our -detailed documentation: [Under the Hood: Contexture -Client - Reactors in Detail](../../under-the-hood/contexture-client.md#reactors-in-detail). - -[↪ Next: Types and Type Components](../../types/README.md) diff --git a/types/README.md b/types/README.md deleted file mode 100644 index ede9f49..0000000 --- a/types/README.md +++ /dev/null @@ -1,13 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - -# Types and Type Components - -Table of Contents: -- [DIY Types](diy-types.md) - - [How to Write a Provider Type](diy-types.md#how-to-wite-a-provider-type) - - [How to Write a Client Type](diy-types.md#how-to-wite-a-client-type) - - [How to Write a UI Component for a Type](diy-types.md#how-to-write-a-ui-component-for-a-type) - - [The Example Types](diy-types.md#the-example-types) -- [ElasticSearch Example Types](elasticsearch-example-types.md) -- [Mongo Example Types](mongo-example-types.md) -- [Available React Components for Types](react-components.md) diff --git a/types/diy-types.md b/types/diy-types.md deleted file mode 100644 index 2c4094c..0000000 --- a/types/diy-types.md +++ /dev/null @@ -1,182 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Types and Type Components](README.md) - -# DIY Types - -The Contexture ecosystem provides a defined list of types that can be -used to perform a wide variety of different searches. Each type we -offer also has a respective component in our `contexture-react` repo. -We've made these components so you can quickstart your search interfaces! - -Even if our types are focused on the different search interfaces we -provide, our API is designed to allow you to build any type you might -need for any other possible use case you might encounter. - -We believe that making a generic framework will allow users to be -creative on their search solutions. Because of that, we will start this -document by explaining how to build your own types. - -Writing a new single type is about writing two plain JavaScript Objects: -- One which is sent to the [Contexture Provider](../querying/available-providers.md). -- Another one which is sent to the initialization of the [Context Tree](../interactive-queries/contexture-client.md#context-tree). - -## How to Wite a Provider Type - -Initializing [Contexture Core](../querying/contexture-core.md) -requires you to send a bunch of types per provider. A type on any -provider is just a valid string property name on the object that is -sent, accompanied with a corresponding value of a plain JavaScript -Object with one or more of the following properties: - -| Property | Type | Params (Type) | Return Value Type | What it does | -| --- | --- | --- | --- | --- | -| `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | -| `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | -| `result` | Function | Node (Object), Search (Function) | Promise | Allows running a direct search for this type before Comtexture sends the full seaech with the whole tree. | - -Once you have written a type, you can use it by sending it to an -existing [Contexture Provider](../querying/available-providers.md). It -should look more or less like: - -```javascript -let myType = { - hasValue: node => node.requiredProperty, - filter: node => ({ - providerQueryObject: { - value: node.requiredProperty - } - }) -} - -let provider = MyProvider({ - types: { - myType - } -}) -``` - -**Type names are not exclusive across providers**. You can define one -type called `myAwesomeType` in more than one provider and you'll be -able to keep the same required node properties, thus the same -`hasValue`. This allows us to provide the same API for several types, -and re-use code even if we switch the target database of the search. - -Once you have a type defined for one or more providers, you should -write the same type for `contexture-client`. - -## How to Write a Client Type - -Contexture Client already provides a some types based on our -`Example Types` (more on that later.) These type definitions help -the client understand how a specific node affects every other node or -itself. - -To create a custom type, you will need to think on the behaviors you -might need for each one of the following properties: - -| Property Name | Type | Description | -| --- | --- | --- | -| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | -| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | -| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | - -More details about `contexture-client` types, their properties and -their reserved words can be seen on the README of -[contexture-client](https://github.com/smartprocure/contexture-client#client-types). - -The example types are already included in any instantiation -of Contexture Client's Contexture Tree. However, you can add any type -you need simply by extending the exposed `exampleTypes` with your own. -In the following snippet, we initialize a `ContextureTree` with the -available `exampleTypes`, and our new `myType`: - -```javascript -import * as ContextureClient from 'contexture-client' - -let tree = ContextureClient.ContextTree({ - types: { - ...ContextureClient.exampleTypes, - myType: { - validate: node => node.requiredProperty, - reactors: { - requiredProperty: 'others', - }, - defaults: { - requiredProperty: false - } - } - } -}, { - // Here we will have the underlying tree -}) -``` - -## How to Write a UI Component for a Type - -Writing a user interface for any type can be as simple as writing an -HTML or JSX Element that will render or modify any property of the -any node of an existing Contexture Tree, for example, using our custom -type `myType`, we could write an input field that, onChange, will -write the field's value into the `requiredProperty`. For example: - -```javascript -// This is ES6+ and JSX - -import * as ContextureClient from 'contexture-client' - -let tree = ContextureClient.ContextTree( - { - service: myService, - types: { - ...ContextureClient.exampleTypes, - myType: { - validate: node => node.requiredProperty, - reactors: { - requiredProperty: 'others', - }, - defaults: { - requiredProperty: false - } - } - } - }, { - key: 'root', - join: 'and', - children: [{ - key: 'myNode', - type: 'myType', - }] - } -) - -let node = tree.getNode(['root', 'myNode']) - -let Component = ({ node }) => ( - { - node.requiredProperty = e.target.value - }} - /> -) -``` - -Now that you have a component you can render it and play with it, but -the component won't render by itself. If you want to see examples of -custom components with automatic updates, please look at our [Managing -State Guide](../managing-state/index.md). - -## The Example Types - -With the intention of providing practical examples of how to write -types, we decided to share some of the types we use in our production -applications. These types belong to two different database processors: -`contexture-elasticsearch` and `contexture-mongo`. - -**Example Types aren't the rule**. These types are only provided to -serve as a guide to build any other type you might need for your -application. You can also **extend our example types**, simply by -assigning new types as properties on the objects exposed by -`contexture-elasticsearch` and `contexture-mongo`. - -[↪ Next: ElasticSearch Example Types](elasticsearch-example-types.md) diff --git a/types/elasticsearch-example-types.md b/types/elasticsearch-example-types.md deleted file mode 100644 index 08d6426..0000000 --- a/types/elasticsearch-example-types.md +++ /dev/null @@ -1,539 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: DIY Types](diy-types.md) - -# ElasticSearch Example Types - -Contexture is designed to target any database you might need. However, -so far we have only inplemented database providers for the only -databases that we use: ElasticSearch and Mongo. - -Most of our types have relevant components already written to -facilitate building search interfaces. As we progress over each one of -these types, we will show small examples of simple components written -to search with these types. We hope to provide enough information to -allow you to take as little as you need, or as much as you want, and -fulfill you expectations. - -Our ElasticSearch types are the following ones: - -## Results Type - -The `results` node is a node that if present, idicates to Contextre -that the queries and aggregations should finally run and retrieve all -the filtered values. It allows some customization properties, as -shown below: - -| Property Name | Type | Description | -| --- | --- | --- | -| `page` | Number | Current page of results. | -| `pageSize` | Number | Total results per page. | -| `sortField` | String | Field used to sort the results. | -| `sortDir` | String | Direction of sorting (`asc`, `desc`) | - -## Bool Type - -The bool type is intended to work as an ElasticSearch terms -aggregation with only one value for a single property. This is useful -for user interfaces with a checkbox to include or exclude a specific -field from a search (or a specific field-value pair). - -Here is the list of all properties this type uses: - -| Property Name | Type | Description | -| --- | --- | --- | -| field | String | Name of the field that we will be using to filter the search search. | -| value | String | Value of the field that will be used to filter the search. | - -Example input: - -```javascript -{ - type: 'bool', - field: 'fieldName', - value: true -} -``` - -Example output: - -```javascript -{ - term: { - fieldName: true - } -} -``` - -You can read more about it in: - -- [Source code of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js). -- [Unit tests of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/bool.js). -- Elastic Search [Term Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html). - -## Cardinality Type - -The Cardinality type serves to calculate an approximate count of the -distinct available values. This type only uses one property, `field`. -It returns it's values in the `context` of the node, rather than in -the `results` node. - -Example input: -```javascript -{ - type: 'cardinality', - field: 'Organization.Name.untouched' -} -``` - -Example output: -```javascript -{ - aggs: { - cardinality: { - cardinality: { - field: 'Organization.Name.untouched' - } - } - } -} -``` - -You can read more about it in: - -- [Source code of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/cardinality.js). -- [Unit tests of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/cardinality.js). -- Elastic Search [Cardinality Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html). - -## Date Type - -The Date type is used to specify a range of dates that will be used to -filter the available results. This range of dates can be specified by -a string formatted date (`YYYY-MM-DD`) or by a small set of possible -humanly readable date ranges. Details follow: - -| Property Name | Type | Description | -| --- | --- | --- | -| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | -| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | -| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | -| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | - -Example input 1: -```javascript -{ - type: 'date', - field: 'fieldName', - from: '2016-04-25' -} -``` - -Example output 1: -```javascript -{ - range: { - fieldName: { - gte: '2016-04-25', - format: 'dateOptionalTime' - } - } -} -``` - -Example input 2: -```javascript -{ - type: 'date', - field: 'fieldName', - from: 'thisQuarter', - useDateMath: true, -} -``` - -Example output 2: -```javascript -{ - range: { - fieldName: { - gte: moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD'), - lte: moment.utc(datemath.parse(`${moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD')}||+3M-1d/d`)).format('YYYY-MM-DD'), - format: 'dateOptionalTime' - } - } -} -``` - -You can read more about it in: - -- [Source code of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). -- [Unit tests of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). -- Elastic Search [Range QUery](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html). - -## Date Histogram Type - -The `dateHistogram` type is very useful for retrieving the number of -results within specific periods of time. The idea here is to end up -building a chart showing how records have changed over time, for -example by it's creation date or some other date property. This type -is able to do this by running a nested stats aggregation inside a -dateHistogram aggregation, while also supporting for tweaking min/max -bounds. - -This type returns it's values in the `context` of the node, rather -than in the `results` node. - -| Property Name | Type | Description | -| --- | --- | --- | -| `key_field` | String | What might be considered a good candidate for the X axis of the chart. | -| `value_field` | String | What might be considered a good candidate for the Y axis of the chart. | -| `interval` | | String | Available expressions for interval: `year` (`1y`), `quarter` (`1q`), `month` (`1M`), `week` (`1w`), `day` (`1d`), `hour` (`1h`), `minute` (`1m`), `second` (`1s`). | -| `boundsRange_min` | Date or String | Lower date limit that will be considered to filter the possible results. | -| `boundsRange_max` | Date or String | Upper date limit that will be considered to filter the possible results. | -| `boundsRange_useDateMath` | Boolean | If set to `true`, the min and max ranges will be valid as strings representative of dates, and will be formatted by NPM's library [`@elastic/datemath`](https://github.com/elastic/datemath-js). | - -Example input: -```javascript -{ - type: 'dateHistogram', - key_field: 'PO.IssuedDate', - value_field: 'LineItem.TotalPrice' -} -``` - -Example output: -```javascript -{ - aggs: { - max_date: { - max: { - field: 'PO.IssuedDate', - }, - }, - min_date: { - min: { - field: 'PO.IssuedDate', - }, - }, - twoLevelAgg: { - date_histogram: { - field: 'PO.IssuedDate', - interval: 'year', - min_doc_count: 0, - }, - aggs: { - twoLevelAgg: { - stats: { - field: 'LineItem.TotalPrice', - }, - }, - }, - }, - }, -} -``` - -You can read more about it in: - -- [Source code of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). -- [Unit tests of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). -- [Two Level Aggregations in ElasticSearch](https://www.elastic.co/blog/intro-to-aggregations-pt-2-sub-aggregations). -- [Date Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html). - -## Exists Type - -The `exists` type is used to check wether some property exsits or not. -It requires only two fields: `field` and `value`. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The target field we want to check. | -| `value` | Boolean | the value we want to check. Normally true or false. | - -Example input: -```javascript -{ - type: 'exists', - field: 'fieldName', - value: true -} -``` - -Example output: - -```javascript -{ - exists: { - field: 'fieldName' - } -} -``` - -You can read more about it in: - -- [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). -- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). -- Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). - -## Facet Type - -The `facet` type represents a list of dynamic choices, e.g. a checkbox -list filter. We achieve this by running an ElasticSearch terms -aggregation. We provide a way to limit the number of results that you -will receive with the `size` property, so that large queries can -safely be used. For that same purpose, the property `optionsFilter` is -given, so that search queries can filter the results with a string. - -Facet returns it's values in the `context` of the node, rather than in -the `results` node. - -| Property | Type | Default | Description | -| ---- | ---- | ------- | ----------- | -| `field` | String | None, *required* | The field it's operating on. | -| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | -| `values` | Array (of strings) | `[]` | Already selected values. | -| `fieldMode` | String (`autocomplete`, `word` or `suggest`) | `autocomplete` | Whether to look at the entire field (`autocomplete`), the analyzed words in the field (`word`), or magic suggestions (`suggest`). | -| `size` | Number | 12 | How many options to return. | -| `cardinality` | Number | 5000 | Precision threshold override. | -| `includeZeroes` | Boolean | false | If true, it will include options with 0 matching documents (aka `min_doc_count: 0`) | -| `optionsFilter` | String | '' | Filters the options further, e.g. a find box above a checkbox list | -| `caseSensitive` | Boolean | false | Whether options filter is case sensitive. | -| `sort` | String (`term` or `count`) | `count` | Sort results alphabetically or by count of matching records. | - -Example input 1: -```javascript -{ - type: 'facet', - field: 'fieldName', - values: ['abc', '123'] -} -``` - -Example output 1: -```javascript -{ - terms: { - 'fieldName.untouched': ['abc', '123'] - } -} -``` - -Example input with exclude: -```javascript -{ - type: 'facet', - field: 'fieldName', - mode: 'exclude', - values: ['abc', '123'], -} -``` - -Example output with exclude: -```javascript -{ - bool: { - must_not: { - terms: { - 'fieldName.untouched': ['abc', '123'], - }, - }, - }, -} -``` - -You can read more about it in: - -- [Source code of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). -- [Unit tests of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). -- Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). - -## Geo Type - -The `geo` type represents a geographic radius search. It needs a -geocodeLocation service passed in to it. We currently assume that you -will be using a google maps geocoder search. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String (required) | The field it's operating on | -| `location` | String (required) | Location to geocode (e.g. an address, businessname, anything the google geocode can take) | -| `radius` | Number (required) | Radius in miles | -| `operator` | String (either `within` or `not within`) | Whether the filter forces inclusion or exclusion (defaults with `within`). | - -Example input: -```javascript -{ - type: 'geo', - field: 'fieldName', - location: 'SmartProcure', - radius: 10, - operator: 'within', -} -``` - -Example output: - -```javascript -{ - geo_distance: { - fieldName: '26.3170479,-80.1131784', - distance: '10mi', - }, -} -``` - -You can read more about it in: - -- [Source code of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/geo.js). -- [Unit tests of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/geo.js). -- [ElasticSearch's Geo Distance Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-query.html). - -## Number Type - -Number represents a number range with inclusive bounds. This type -provides the ability to determine the best range values based on -percentile interval and range threshold. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `min` | Number | Lower boundary of the filter. | -| `max` | Number | Upper boundary of the filter. | - -Some Notes: -1. An empty value as the upper boundary represents infinity. -2. An empty value as the lower boundary represents negative infinity. -3. Zero has to be respected as a boundary value. -4. If findBestRange is true it will return the best min and max range. - -Example input: -```javascript -{ - type: 'number', - field: 'fieldName', - min: 500, - max: 1000, -} -``` - -Example output: - -```javascript -{ - range: { - fieldName: { - gte: 500, - lte: 1000, - }, - }, -} -``` - -You can read more about it in: - -- [Source code of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). -- [Unit tests of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). - -## Number Range Histogram Type - -The type `numberRangeHistogram` represents a number range with inclusive bounds. This type -returns feedback in the form of histogram and statistical data. -This type returns it's values in the `context` of the node, rather -than in the `results` node. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `min` | Number | Lower boundary of the filter. | -| `max` | Number | Upper boundary of the filter. | -| `percentileInterval` | Number | Used to group the results based on how many of the records are within each one of the sections given by the interval, from the `min` value, up to the `max` value. | - -Some Notes: -1. An empty value as the upper boundary represents infinity. -2. An empty value as the lower boundary represents negative infinity. -3. Zero has to be respected as a boundary value. - -- [Source code of the type: numberRangeHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/numberRangeHistohgram.js). -- [Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/search-aggregations-bucket-histogram-aggregation.html). - -## Query Type - -Query represents a raw elasticsearch query_string. It's mostly used to -provide a simple sarch box on the user interface. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | -| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches (accepting values if they contain the given query, even if the casing doesn't match). | - -Example input: -```js -{ - type: 'query', - field: 'fieldName', - query: 'cable', - exact: true, -} -``` - -Example output: -```js -{ - query_string: { - query: 'cable', - default_operator: 'AND', - default_field: 'fieldName.exact', - analyzer: 'exact', - }, -} -``` - -You can read more about it in: - -- [Source code of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). -- [Unit tests of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). - -## Text Type - -Text implements raw text analysis like starts with, ends with, etc. -These are generally regex queries. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `join` | String | Either `any`, `all` or `none`. | -| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | -| `values` | Array | Array containing all the words that want to be used as inputs. | - -Example input: -```js -{ - type: 'text', - field: 'fieldName', - join: 'any', - operator: 'contains', - values: ['laserjet', 'printer'], -} -``` - -Example output: -```js -{ - query_string: { - default_field: 'fieldName', - default_operator: 'OR', - query: '"laserjet" "printer"', - }, -} -``` - -You can read more about it in: - -- [Source code of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/text.js). -- [Unit tests of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/text.js). - -## Other ElasticSearch Example Types - -For more informaion about other available example types, please check: - - -[↪ Next: Mongo Example Types](mongo-example-types.md) diff --git a/types/mongo-example-types.md b/types/mongo-example-types.md deleted file mode 100644 index 590e1ae..0000000 --- a/types/mongo-example-types.md +++ /dev/null @@ -1,251 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: ElasticSearch Example Types](elasticsearch-example-types.md) - -# MongoDb Example Types - -Most of our types have relevant components already written to -facilitate building search interfaces. As we progress over each one of -these types, we will show small examples of simple components written -to search with these types. We hope to provide enough information to -allow you to take as little as you need, or as much as you want, and -fulfill you expectations. - -Our MongoDb types are the following ones: - -## Results Type - -The `results` node is a node that if present, idicates to Contextre -that the queries and aggregations should finally run and retrieve all -the filtered values. It allows some customization properties, as -shown below: - -| Property Name | Type | Description | -| --- | --- | --- | -| `page` | Number | Current page of results. | -| `pageSize` | Number | Total results per page. | -| `sortField` | String | Field used to sort the results. | -| `sortDir` | String | Direction of sorting (`asc`, `desc`) | - -## Date Type - -The Date type is used to specify a range of dates that will be used to -filter the available results. This range of dates can be specified by -a string formatted date (`YYYY-MM-DD`) or by a small set of possible -humanly readable date ranges. Details follow: - -| Property Name | Type | Description | -| --- | --- | --- | -| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | -| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | -| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | -| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | - -Example input: -```javascript -{ - type: 'date', - field: 'fieldName', - from: '2016-04-25' -} -``` - -Example output: -```javascript -{ - fieldName: { - $gte: new Date('2016-04-25'), - }, -} -``` - -You can read more about it in: - -- [Source code of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). -- [Unit tests of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). -- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). -- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). - -## Exists Type - -The `exists` type is used to check wether some property exsits or not. -It requires only two fields: `field` and `value`. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The target field we want to check. | -| `value` | Boolean | the value we want to check. Normally true or false. | - -Example input: -```javascript -{ - type: 'exists', - field: 'fieldName', - value: true -} -``` - -Example output: - -```javascript -{ - $and: [ - { - fieldName: { - $exists: true, - $ne: '', - }, - }, - { - fieldName: { - $ne: null, - }, - }, - ], -} -``` - -You can read more about it in: - -- [Source code of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/exists.js). -- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/exists.js). -- MongoDb's [$exists](https://docs.mongodb.com/manual/reference/operator/query/exists/). - -## Facet Type - -The `facet` type represents a list of dynamic choices, e.g. a checkbox -list filter. We achieve this by running an ElasticSearch terms -aggregation. We provide a way to limit the number of results that you -will receive with the `size` property, so that large queries can -safely be used. For that same purpose, the property `optionsFilter` is -given, so that search queries can filter the results with a string. - -Facet returns it's values in the `context` of the node, rather than in -the `results` node. - -| Property | Type | Default | Description | -| ---- | ---- | ------- | ----------- | -| `field` | String | None, *required* | The field it's operating on. | -| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | -| `values` | Array (of strings) | `[]` | Already selected values. | -| `size` | Number | 10 | How many options to return. | - -Example input: -```javascript -{ - type: 'facet', - field: 'fieldName', - values: ['abc', '123'] -} -``` - -Example input with exclude: -```javascript -{ - type: 'facet', - field: 'fieldName', - mode: 'exclude', - values: ['abc', '123'], -} -``` - -You can read more about it in: - -- [Source code of the type: facet](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). - -## Number Type - -Number represents a number range with inclusive bounds. This type -provides the ability to determine the best range values based on -percentile interval and range threshold. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `min` | Number | Lower boundary of the filter. | -| `max` | Number | Upper boundary of the filter. | - -Some Notes: -1. An empty value as the upper boundary represents infinity. -2. An empty value as the lower boundary represents negative infinity. -3. Zero has to be respected as a boundary value. -4. If findBestRange is true it will return the best min and max range. - -Example input: -```javascript -{ - type: 'number', - field: 'fieldName', - min: 500, - max: 1000, -} -``` - -Example output: - -```javascript -{ - fieldName: { - $gte: 500, - $lte: 1000 - } -} -``` - -You can read more about it in: - -- [Source code of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). -- [Unit tests of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). -- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). -- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). - -## Text Type - -Text implements raw text analysis like starts with, ends with, etc. -These are generally regex queries. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `join` | String | Either `any`, `all` or `none`. | -| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | -| `values` | Array | Array containing all the words that want to be used as inputs. | - -Example input: -```js -{ - type: 'text', - field: 'fieldName', - join: 'any', - operator: 'contains', - values: ['laserjet', 'printer'], -} -``` - -Example output: -```js -{ - $or: [{ - fieldName: { - $regex: 'laserjet', - $options: 'i' - } - }, { - fieldName: { - $regex: 'printer', - $options: 'i' - } - }] -} -``` - -You can read more about it in: - -- [Source code of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/text.js). -- [Unit tests of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/text.js). - -## Other MongoDb Example Types - -For more informaion about other available example types, please check: - - -[↪ Next: Available React Components for Types](react-components.md) diff --git a/types/react-components.md b/types/react-components.md deleted file mode 100644 index 2804406..0000000 --- a/types/react-components.md +++ /dev/null @@ -1,401 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) -[↩ Previous: Mongo Example Types](mongo-example-types.md) - -# Available React Components for Types - -A huge part of working with advanced search interfaces is identifying -how to portray a specific search filter that you might want to use. -We've talked about very simple components that react to changes on the -tree state, but now we'll see how to make it easier for more complex -nodes to gather the inputs necessary, and also to show the results -correctly. Here we'll see components specifically crafted for some of -our providers' example types. - -**Notes:** -- Keep in mind that the theme on these components is purely -optional. Later on we'll see how to [theme our -components](../theming/README.md). -- Please be aware that when we refer to `Component`, we mean a - Function that returns a valid JSX Element. You can read more here: - [Components and - Props, on the ReactJS docs](https://reactjs.org/docs/components-and-props.html). - -## Query - -![Query Type Screenshot](https://i.imgur.com/8r2X9MI.png) - -The Query component is probably the most commonly needed for search -interfaces. It's an input field that allows you to filter results if -the text matches part of the value of a specfic property on any of the -records. - -Here's how you write a node of type `query` in your _searchTree_: -```javascript -{ - key: 'searchQuery', - type: 'query', - field: 'title', - query: '' -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `query` | String | No | Search query that should be visible in the component. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. See the Provider type's -documentation in our [previous pages](README.md). - -Here's how you write your component: -```javascript -let Query = require('contexture-react/dist/exampleTypes').Query -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `TextInput` | Component | `input` | Text input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). -- [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). -- [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). -- [(MongoDb Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/query.js). -- [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). - -## Date - -![Date Type Screenshot 1](https://i.imgur.com/XwuGi2c.png) -![Date Type Screenshot 2](https://i.imgur.com/joTECy0.png) - -_(Same goes with the right, not adding another screenshot to avoid -consuming more space.)_ - -The Date component helps by allowing users to filter the data to -obtain results within a specific range of dates. It consists of only -two date pickers. Here's how you write a node of type `date` in your -_searchTree_: -```javascript -{ - type: 'date', - field: 'fieldName', - from: '2016-04-25', - to: '2017-05-26' -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `from` | Date or String (`YYYY-MM-DD` format) | Yes | The initial date of our timeframe. | -| `to` | Date or String (`YYYY-MM-DD` format) | No | The final date of our timeframe. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. See the Provider type's -documentation in our [previous pages](README.md). - -Here's how you write your component: -```javascript -let DateComponent = require('contexture-react/dist/exampleTypes').Date -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `DateInput` | Component | `x => ` | The component that wraps each one of the inputs where the dates end up written by the date-picker. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). -- [(ElasticSearch Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). -- [(MongoDb Provider) Source code of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). -- [(MongoDb Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). -- [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). - -## DateHistogram - -![DateHistogram Screenshot](https://i.imgur.com/oZsXY5R.png) - -The DateHistogram component is about representing how many records -were found during what periods of time. This component currently -doesn't offer interactive features, but you could use it as -inspiration to write another one that would allow you to dive in these -date ranges to see what records appear for each one of them. - -Here's how you write a node of type `dateHistogram` in your _searchTree_: -```javascript -{ - key: 'releases', - type: 'dateHistogram', - key_field: 'released', - value_field: 'imdbVotes', - interval: '3650d', -} -``` - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. See the Provider type's -documentation in our [previous pages](README.md). - -Since this component doesn't need any property from the node itself, -but only from the results, here's how you write your component: -```javascript -let DateHistogram = require('contexture-react/dist/exampleTypes').DateHistogram -let formatYear = x => new Date(x).getFullYear() + 1 -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `background` | Function | `(record, max) => '#ccc'` | A function that returns the background color that is used to render each one of the bars in the resulting chart. | -| `height` | Number | `100` | Specifies the max height of the whole chart. | -| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | -| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | -| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). -- [(ElasticSearch Provider) Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). -- [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). - -## Facet - -![Facet Type Screenshot](https://i.imgur.com/1X3mrfq.png) - -The Facet component allows users to filter the results by picking a -specific common option among all the values that the records might -have for a specific field. - -Here's how you write a node of type `facet` in your _searchTree_: -```javascript -{ - key: 'facet', - type: 'facet', - values: ['a'], -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `values` | Array | Yes | Array of selected values. To have a value selected by default, you will need to know in advance which value to put here, otherwise you can leave this with an empty array and let users select the values themselves. | -| `size` | Number | No | Max number of options to display. The default is 10. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. See the Provider type's -documentation in our [previous pages](README.md). - -Here's how you write your component: -```javascript -let Facet = require('contexture-react/dist/exampleTypes').Facet -// ... -// Later, on your render function, or where you put your components: - -``` - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `hide.facetFilter` | Object (with an optional single property, `facetFilter`) | `{}` | Allows you to hide the text input that helps on searching for the available options. This text input is very valuable when the results are larger than the available visible results. | -| `TextInput` | Component | `input` | Allows you to customize the text input. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). -- [(ElasticSearch Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). -- [(MongoDb Provider) Source code of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). -- [(MongoDb Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/facet.js). -- [Source code of the Facet component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Facet.js). - -## Number - -![Number Type Screenshot](https://i.imgur.com/Uuu16wy.png) - -The Number component allows users to specify a numeric range to filter -the data based on the available results which values fit within this -range for a specific field. - -Here's how you write a node of type `number` in your _searchTree_: -```javascript -{ - key: 'searchNumber', - type: 'number', - field: 'metaScore', - min: 0, - max: 100, -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minimum number for the range. | -| `max` | Number | No | Maximum number for the range. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. See the Provider type's -documentation in our [previous pages](README.md). - -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/exampleTypes').Number -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `NumberInput` | Component | `x => ` | Number input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). -- [(ElasticSearch Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). -- [(MongoDb Provider) Source code of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). -- [(MongoDb Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). -- [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). - -## ResultCount - -![ResultCount Type Screenshot](https://i.imgur.com/htuXprD.png) - -The ResultCount component will only show you the number of visible -results compared to the number of total results. It's not an -interactive component. - -Most of your Contexture Trees will have a node with type `results`. -This node posesses information such as the resulting records -themselves, but also which page you're in, how many -elements are per page you will receive, and the total records there -are for this given query, which is what we're looking for for this -type. - -All of these properties are automatically writen by the Contexture -architecture, so we'll be simply passing the tree and the path to the -results type node: - -```javascript -let ResultCount = require('contexture-react/dist/exampleTypes').ResultCount -// ... -// Later, on your render function, or where you put your components: - -``` - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). -- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). -- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). -- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). -- [Source code of the ResultCount component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultCount.js). - -## ResultPager - -![ResultPager Type Screenshot](https://i.imgur.com/h73I2QZ.png) - -The ResultPager component is an interactive component that will show -you which page you're at (in the middle) and what pages are around -your current page, as well as some controllers to move forward and -backwards. - -The style of this component isn't very friendly, but it's very easy to -customize. For more about theme changes, please visit our -[theming docs](../theming/README.md). - -Most of your Contexture Trees will have a node with type `results`. -This node posesses information such as the resulting records -themselves, but also which page you're in, how many -elements are per page you will receive, and the total records there -are for this given query. We use a combination of those values to get -how many pages we can move forward (and backwards), and also to move -around these pages (since our trees react in real time). - -All of these properties are automatically writen by the Contexture -architecture, so we'll be simply passing the tree and the path to the -results type node: - -```javascript -let ResultPager = require('contexture-react/dist/exampleTypes').ResultPager -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `List` | Component | `div` | The component that wraps the whole list of pages. | -| `Item` | Component | `span` | The component that wraps each of the available page numbers. | -| `Link` | Component | `a` | An element that wraps each one of the pagination controls. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). -- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). -- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). -- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). -- [Source code of the ResultPager component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultPager.js). - -## ResultTable - -![resulttable type screenshot](https://i.imgur.com/cc5urub.png) - -The ResultTable is a component that will display a table with all the -available results, in which each of the result values will be -displayed as columns. - -the results are automatically writen by the contexture architecture, -so we'll be simply passing the tree and the path to the results type -node: - -```javascript -let resulttable = require('contexture-react/dist/exampletypes').resulttable -// ... -// later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `Table` | Component | `table` | The component that wraps the table list of results. | - -To read more, check the following links: - -- [(ElasticSearch Provider) source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). -- [(elasticSearch Provider) unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). -- [(MongoDb Provider) source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). -- [(MongoDb Provider) unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). -- [Source code of the ResultTable component](https://github.com/smartprocure/contexture-react/blob/master/src/exampletypes/ResultTable.js). - -[↪ Next: Other Components](../other-components/README.md) diff --git a/under-the-hood/README.md b/under-the-hood/README.md deleted file mode 100644 index 5213f1b..0000000 --- a/under-the-hood/README.md +++ /dev/null @@ -1,21 +0,0 @@ -[↩ Parent: Table of Contents](../README.md) - - # Under the Hood - -Table of Contents: -- [Design Principles](design-principles.md) -- [Contexture Core](contexture-core.md) - - [Default Export](contexture-core.md#default-export) - - [The Algorithm](contexture-core.md#the-algorithm) - - [Utility Functions](contexture-core.md#utility-functions) -- [Contexture Providers](contexture-providers/README.md) - - [Contexture ElasticSearch](contexture-providers/contexture-elasticsearch.md) - - [Contexture Mongo](contexture-providers/contexture-mongo.md) - - [Building Your Own Provider](contexture-providers/building-your-own-provider.md) -- [Contexture Client](contexture-client.md) - - [State Properties](contexture-client.md#state-properties) - - [Lenses](contexture-client.md#lenses) - - [Reactors in Detail](contexture-client.md#reactors-in-detail) - - [Serialization](contexture-client.md#serialization) - - [Traversals](contexture-client.md#traversals) -- [Contexture React](contexture-react.md) diff --git a/under-the-hood/contexture-client.md b/under-the-hood/contexture-client.md deleted file mode 100644 index 6652437..0000000 --- a/under-the-hood/contexture-client.md +++ /dev/null @@ -1 +0,0 @@ -# Contexture Client diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md deleted file mode 100644 index 4703eb9..0000000 --- a/under-the-hood/contexture-core.md +++ /dev/null @@ -1,60 +0,0 @@ -# Contexture Core - -The core of Contexture is a package of its own. Located at [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture), it offers the very underlying function that is designed to process every one of the search queries. It begins with a simple curried function which, once the providers and the schemas are received, proceeds to process the [Contexture DSL](#TODO) by walking down the given search tree, cleaning up every node of possible inconsistencies, then mutating the tree with the directions given by the provider types and schemas, up until a valid search query is obtained. This query is delivered to the provider `runSearch` method, and the result is finally added back to the tree. - -With this in mind, let's get some specifications. - -## Default Export - -Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. - -The first argument is expected to be a plain JavaScript Object with two keys: - -- `providers`: Should be an object where each key will have a [Contexture Provider](#TODO). -- `schemas`: Should be an object where each key will have a [Contexture Schema](#TODO). - -Calling this function with this object only will return another function, which can be used as an asynchronous search runner. You can also pass in all the arguments as once, but the separation of parameters makes it easier to scope setting up the database providers, the types and the schemas from the search execution. - -Example declaration of a search function by passing the schema & providers object first: - -```javascript -const search = Contexture({ - schemas: { - ...mongoSchemas, - ...elasticSearchSchemas, - }, - providers: { - mongo: require('contexture-mongo')({ /* provider configuration */ }), - elasticsearch: require('contexture-elasticsearch')({ /* provider configuration */}), - }, -}) - -// How you might use it with an express-like API: -// app.use('/search', async req, res) => -// res.send(await search(req.body.search)) -``` - -The other two parameters are the search tree (the [Contexture -DSL](#TODO)), and an optional object that will be sent along to the -provider's `runSearch` function as the first parameter, that can -contain any property, but that should at least contain the following -properties: - -| Option | Description | -| ------ | ----------- | -| `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info. | -| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example. | - -This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the [Contexture Types](#TODO). - -## The Algorithm - -_Coming soon..._ - -- Initial values in the tree. -- Flat lenses. -- State flags. -- Add filters. -- DFS initialization. - -## Utility Functions diff --git a/under-the-hood/contexture-providers/building-your-own-provider.md b/under-the-hood/contexture-providers/building-your-own-provider.md deleted file mode 100644 index c8ba146..0000000 --- a/under-the-hood/contexture-providers/building-your-own-provider.md +++ /dev/null @@ -1,92 +0,0 @@ -[↩ Parent: Under the Hood](../README.md) -[↩ Previous: Contexture Mongo](contexture-mongo.md) - -# Building Your Own Provider - -Writing a new provider consists of writing a function (or class or -any other approach you might like), that will receive a configuration -object with at least the following properties: - -| Property Name | Type | Description | -| --- | --- | --- | -| `types` | `Object` | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo). More information on our detailed docs about [Contexture Types](../types/diy-types.md) | -| `getClient` | `Function` | A function that returns the main database client that you will be using. | - -You might also add any new property if your provider needs it for -processing the searches. - -The main provider function is expected to return an Object with: - -| Property Name | Type | Description | -| --- | --- | --- | -| `types` | `Object` | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo). More information on our detailed docs about [Contexture Types](../types/diy-types.md) | -| `groupCombinator(group, filters)` | `Function` | A function that returns a valid query for grouping individual aggregations. More on that below. | -| `runSearch(options = {}, context, schema, filters, aggs)` | `Function -> Promise` | A function that will be responsible for running each one of the individual searches that the types in the Contexture DSL might indicate. More on that below. | - -### Group Combinator Function - -The `groupCombinator` function is used to transform an incoming query -into the underlying database's group combinator. It receives a `group` -Object that contains a property `join`, which indicates which boolean -operation wants to be used. The `join` values we use are `and`, `or` -and `not`, but you can specify any `join` value for your provider, as -long as you make sure the resulting query is valid. The result should -be a native database query structure with the given `filters` within a -boolean operator. For example, in ElasticSearch, this means: - -| Boolean Operation | ElasticSearch Combinator | -| --- | --- | -| `and` | `{ bool: { must: filters } }` | -| `or` | `{ bool: { should: filters, minimum_should_match: 1 } }` | -| `not` | `{ bool: { must_not: filters } }` | - -However, in MongoDB, this is how it works: - -| Boolean Operation | MongoDB Combinator | -| --- | --- | -| `and` | `{ $and: filters }` | -| `or` | `{ $or: filters }` | -| `not` | `{ $nor: filters }` | - -### Run Search Function - -The `runSearch` function will run each search that is indicated by the -Contexture DSL. It receives several parameters: - -| Property Name | Type | Description | -| --- | --- | --- | -| `options` | `Object` | Object that is sent as the last parameter to the main `Contexture` call. | -| `context` | `Object` | The current node that is triggering the search. | -| `schema` | `Object` | The schema that is related to this node or root. Usually assigned at the root of the search tree. | -| `filters` | `Array` | The filters that have resulted from parsing this node and it's children. | -| `aggs` | `Array` | Any other aggregation that might have been received from previous processing operations of the Contexture DSL. | - -In this function, you should call to the `getClient` function that the -provider originally received, then you should accomodate the incoming -`filters` and `aggs` into the final query that will be sent to the -client's aggregation method. Since you'll have the schema, you'll be -able to get the information you need for your specific database with -`schema.myDatabase.myProperty`. - -The final search query should be an Object and should be pushed into -the `context._meta.requests` array, for debugging purposes. Once you have -the result, you should also add it to the final query object, so that -it can be accessible by reading the context path, then -`_meta.requests[index].response`, where `index` is the position where -the final query lives. For example: - -```javascript -let request = { - /* My Database DSL Object */ -} - -context._meta.requests.push(request) - -let result = Promise.resolve(databaseClient(request)) - -return result.tap(results => { - request.response = results -}) -``` - -[↪ Next: Contexture Client](../contexture-client.md) diff --git a/under-the-hood/contexture-providers/contexture-elasticsearch.md b/under-the-hood/contexture-providers/contexture-elasticsearch.md deleted file mode 100644 index d7a3556..0000000 --- a/under-the-hood/contexture-providers/contexture-elasticsearch.md +++ /dev/null @@ -1 +0,0 @@ -# Contexture ElasticSearch diff --git a/under-the-hood/contexture-providers/contexture-mongo.md b/under-the-hood/contexture-providers/contexture-mongo.md deleted file mode 100644 index ec8a9c0..0000000 --- a/under-the-hood/contexture-providers/contexture-mongo.md +++ /dev/null @@ -1 +0,0 @@ -# Contexture Mongo diff --git a/under-the-hood/contexture-react.md b/under-the-hood/contexture-react.md deleted file mode 100644 index 32f6e8d..0000000 --- a/under-the-hood/contexture-react.md +++ /dev/null @@ -1 +0,0 @@ -# Contexture React diff --git a/under-the-hood/design-principles.md b/under-the-hood/design-principles.md deleted file mode 100644 index 8fb0b8c..0000000 --- a/under-the-hood/design-principles.md +++ /dev/null @@ -1,25 +0,0 @@ -# Design Principles - -- Intentionally stateless. -- Detached from state management tools. -- Can work well with state management tools. -- Modern ES6+ -- Non-strict functional programming. -- Configuration based architecture. -- Focus on a very extensible small core. -- Small DSL. - - Database agnostic. - - Isomorphic Tree State. - - Optimized for database discovery. - - Optimized for advanced search interfaces. - - Aiming to be effective for arbitrarily complex database indexes. - - Simplicity over performance. -- Reaction management. - - Tree walking. - - State flags. - - What updates. - - Pauses. -- Why types? -- Why a results type? -- Why schemas? -- Why providers? From 9e815d067c9163e048ecac59ded6cf3984e4a93a Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:41:52 -0500 Subject: [PATCH 106/150] clean up readme --- README.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/README.md b/README.md index 5727eae..4a56a9b 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,4 @@ -# Contexture Documentation - -This repo is designed to host the documentation of Contexture as a -whole. Everythin we currently have here is **WORK IN PROGRESS**. - -## About Contexture +## About Contexture ### What is Contexture ### Glossary of Terms ### Map of Repos From 74b05694b6854e74f4f191b913326f18fcb9b695 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 17:42:40 -0500 Subject: [PATCH 107/150] fix md formatting bug --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 4a56a9b..ee997d0 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,6 @@ -## About Contexture +This is the complete documentation for [Contexture](https://github.com/smartprocure/contexture). + +## About Contexture ### What is Contexture ### Glossary of Terms ### Map of Repos From 86bf0d6ea760205f22c59a450dde3671a8e6dc93 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 18:52:56 -0500 Subject: [PATCH 108/150] add first sections docs --- README.md | 200 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 200 insertions(+) diff --git a/README.md b/README.md index ee997d0..1f85805 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,210 @@ ## About Contexture ### What is Contexture +People of the Internet, here we officialy introduce you to +`contexture`, our framework for building search interfaces. + +This framework is carefully designed to be a generic solution for a +universe of unlimited possible search interfaces. We've started with a +minimal set of repositories that are representative of tools that +empower our business, but are intended to be merely examples. If +anything, our approaches are only use cases, for the potential of +this tool is ultimately yours to take. + +A quick search over the Internet would reveal that the word +`contexture` means: `the fact or manner of being woven or linked +together to form a connected whole` and also `the putting together of +words and sentences in connected composition; the construction of a +text`. + +Picking `contexture` as the name for this project means that we are +trying to expose not only our ultimate intentions, but also more or +less how the system is built. The way our projects work is by a DSL +that is used to gather different intended search inputs, each one +representing some useful abstraction of a search filter (like an input +where you can write a word to be searched, or another where you can +filter the search results by one or more options), then using the +values to process a DSL that will end up retrieving values from one or +more different databases, then returning these values on the +respective sections of the DSL, so that each result can update each +one of the components of the user interface. A more detailed +description is visible in the following diagram. + +

+ +The canonical example of a Contexture Node is faceted search, where +you have a checkbox list that is both a filter (in the sense that it +restricts results based on the checked values) and an aggregation +(which shows the top n values that can be checked). Contexture allows +them to be nested in advanced searches with boolean joins like +`and`/`or`/`not`. + +

+ +This thought process will become more clear as we progress through the +docs. Hopefully, some pages later it will be easy to grasp how we +provide a new perspective on building search interfaces, and perhaps +even how you can use it to power up your business, just like we have +been doing for almost a decade. + ### Glossary of Terms +#### Contexture + +> NOUN +> 1. The fact or manner of being woven or linked together to form a +> connected whole. +> 1.1. A mass of things interwoven together; a fabric. +> 1.2. The putting together of words and sentences in connected +> composition; the construct of a text. +> 1.3. A connected literary structure; a continuous text. + +[Source](https://en.oxforddictionaries.com/definition/us/contexture). + +#### Domain-Specific Language (DSL) + +> A domain-specific language (DSL) is a computer language specialized +> to a particular application domain. This is in contrast to a +> general-purpose language (GPL), which is broadly applicable across +> domains. + +[Source](https://en.wikipedia.org/wiki/Domain-specific_language). + +#### Search Query + +According to Wikipedia, `Search Query`, in the context of the `web`, refers to: + +> ...a query that a user enters into a web search engine to satisfy his or her +> information needs. Web search queries are distinctive in that they are often +> plain text or hypertext with optional search-directives (such as "and"/"or" +> with "-" to exclude). They vary greatly from standard query languages, which +> are governed by strict syntax rules as command languages with keyword or +> positional parameters. + +[Source](https://en.wikipedia.org/wiki/Web_search_query). + +#### Faceted Search + +> Faceted search, also known as faceted navigation or faceted browsing, is a +> technique used by eCommerce brands to help users analyze, organize, and +> filter large sets of product inventory based on filters such as size, color, +> price, and brand. + +[Source](https://www.dynamicyield.com/glossary/faceted-search/). + +> Faceted search, also called faceted navigation or faceted browsing, is a +> technique for accessing information organized according to a faceted +> classification system, allowing users to explore a collection of information +> by applying multiple filters. A faceted classification system classifies each +> information element along multiple explicit dimensions, called facets, +> enabling the classifications to be accessed and ordered in multiple ways +> rather than in a single, pre-determined, taxonomic order. + +[Source](https://en.wikipedia.org/wiki/Faceted_search). + +#### Search Filter + +> An extension of faceted search, a search filter is a specific product +> attribute a visitor can use to refine the search results of a particular +> category listing, e.g. by size, color, price, or brand. Multiple filters may +> be applied to take a broad range of products and refine them into a more +> narrow selection, allowing the end user to retrieve the most relevant search +> results based on the criteria they’ve selected. + +[Source](https://www.dynamicyield.com/glossary/search-filter/). + +#### Configuration Based Development + +> The difference between configuration-driven development and model-driven +> development is that the former is not restricted to the model of the code +> such as classes, fields, and relationships. Configuration-driven development +> (CCD) encompasses anything that can be configured within your application. +> For example, if your architecture dictates that particular business rules +> must be applied consistently across your application, you can use +> configuration files to configure and apply those rules. + +[Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). + +#### Functional Logic Programming + +> Functional logic programming is the combination, in a single programming +> language, of the paradigms of functional programming (including higher-order +> programming) and logic programming (nondeterministic programming, +> unification). This style of programming is embodied by various programming +> languages, including Curry and Mercury. + +[Source](https://en.wikipedia.org/wiki/Functional_logic_programming). + ### Map of Repos + +The Contexture framework comes to life through a list of repositories +that individually specialize in some needed layer for our +architecture. We will be using most (if not all) of these projects in +our upcoming pages. Let's explore the repos we have so far: + +#### Contexture Core + +[github.com/smartprocure/contexture](https://github.com/smartprocure/contexture) +is where our main DSL processor lives. It is a very small layer that +ties everything together. This one receives the information about the +different search representations, about the databases involved, and +the DSL, then outputs the search results respective to each one of the +queries described in a copy of the received DSL. + +You can read more about the core here: +- [In the repository](https://github.com/smartprocure/contexture). +- In our [docs about querying (Contexture Core section)](../querying/contexture-core.md). +- Or with greater detail in our [under the hood docs (Contexture Core section)](../under-the-hood/contexture-core.md). + +#### Contexture Providers + +The Contexture Providers are the interfacing layer that ties the +Contexture DSL to the targeted databases. So far, we have only two +open source providers: + +- [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch), + and +- [contexture-mongo](https://github.com/smartprocure/contexture-mongo). + +If you are planning to use either ElasticSearch or MongoDB for your +project, the best way to get started is to use those repositories. +However, if you need to use Contexture with another database, you will +need to implement the provider yourself. We're looking for code +contributors, so please don't feel limited to the current available +tools. Help us grow together! + +You can read more about Contexture Providers here: +- In our [docs about querying (Available Providers section)](../querying/available-providers.md). +- In with greater detail in our [under the hood docs (Contexture Providers section)](../under-the-hood/contexture-providers/README.md). + +#### Contexture Client + +The Contexture Client is responsible for triggering behaviors on the +search interfaces by knowing what causes changes in one or more +elements of the search tree. It is the key piece of technology that +allows our search interfaces to work in real time. + +You can read more about the Contexture Client here: +- [In the repository](https://github.com/smartprocure/contexture-client). +- In our [docs about querying (Contexture Client section)](../interactive-queries/contexture-client.md). +- Or with greater detail in our [under the hood docs (Contexture Client section)](../under-the-hood/contexture-client.md). + +#### Contexture React + +The Contexture React repository holds a list of components that +facilitate building search interfaces. They are mainly graphical +representations of the types that exist on our Contexture Providers. + +You can read more about the Contexture React: +- [In the repository](https://github.com/smartprocure/contexture-client). +- In our guide for the [Available React Components for Types](./types/react-components.md). +- Or in greater detail in our [under the hood docs (Contexture React section)](under-the-hood/contexture-react.md). + + ### Brief History +**TODO** + ### Alternatives & Benchmarks +**TODO** ## Getting Started ### Project Setup From b3f3ddd687e1f55c65e99713705988e9c5573d18 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 18:55:25 -0500 Subject: [PATCH 109/150] remove theme --- docs/index.html | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/index.html b/docs/index.html index 4aabe15..fa76053 100644 --- a/docs/index.html +++ b/docs/index.html @@ -26,8 +26,6 @@ - - From cd964081598edcb51025111b1941a33861a37ff6 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 18:57:47 -0500 Subject: [PATCH 110/150] add custom theme files --- docs/index.html | 3 + docs/script.js | 243 ++++++++++++++ docs/style.css | 878 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 1124 insertions(+) create mode 100644 docs/script.js create mode 100644 docs/style.css diff --git a/docs/index.html b/docs/index.html index fa76053..a08010b 100644 --- a/docs/index.html +++ b/docs/index.html @@ -26,6 +26,9 @@ + + + diff --git a/docs/script.js b/docs/script.js new file mode 100644 index 0000000..7ba961e --- /dev/null +++ b/docs/script.js @@ -0,0 +1,243 @@ +(function($) { + var $window = $(window); + var $document = $(document); + + /* + * Scrollspy. + */ + + $document.on('flatdoc:ready', function() { + $("h2, h3").scrollagent(function(cid, pid, currentElement, previousElement) { + if (pid) { + $("[href='#"+pid+"']").removeClass('active'); + } + if (cid) { + $("[href='#"+cid+"']").addClass('active'); + } + }); + }); + + /* + * Anchor jump links. + */ + + $document.on('flatdoc:ready', function() { + $('.menu a').anchorjump(); + }); + + /* + * Title card. + */ + + $(function() { + var $card = $('.title-card'); + if (!$card.length) return; + + var $header = $('.header'); + var headerHeight = $header.length ? $header.outerHeight() : 0; + + $window + .on('resize.title-card', function() { + var windowWidth = $window.width(); + + if (windowWidth < 480) { + $card.css('height', ''); + } else { + var height = $window.height(); + $card.css('height', height - headerHeight); + } + }) + .trigger('resize.title-card'); + }); + + /* + * Sidebar stick. + */ + + $(function() { + var $sidebar = $('.menubar'); + var elTop; + + $window + .on('resize.sidestick', function() { + $sidebar.removeClass('fixed'); + elTop = $sidebar.offset().top; + $window.trigger('scroll.sidestick'); + }) + .on('scroll.sidestick', function() { + var scrollY = $window.scrollTop(); + $sidebar.toggleClass('fixed', (scrollY >= elTop)); + }) + .trigger('resize.sidestick'); + }); + +})(jQuery); +/*! jQuery.scrollagent (c) 2012, Rico Sta. Cruz. MIT License. + * https://github.com/rstacruz/jquery-stuff/tree/master/scrollagent */ + +// Call $(...).scrollagent() with a callback function. +// +// The callback will be called everytime the focus changes. +// +// Example: +// +// $("h2").scrollagent(function(cid, pid, currentElement, previousElement) { +// if (pid) { +// $("[href='#"+pid+"']").removeClass('active'); +// } +// if (cid) { +// $("[href='#"+cid+"']").addClass('active'); +// } +// }); + +(function($) { + + $.fn.scrollagent = function(options, callback) { + // Account for $.scrollspy(function) + if (typeof callback === 'undefined') { + callback = options; + options = {}; + } + + var $sections = $(this); + var $parent = options.parent || $(window); + + // Find the top offsets of each section + var offsets = []; + $sections.each(function(i) { + var offset = $(this).attr('data-anchor-offset') ? + parseInt($(this).attr('data-anchor-offset'), 10) : + (options.offset || 0); + + offsets.push({ + id: $(this).attr('id'), + index: i, + el: this, + offset: offset + }); + }); + + // State + var current = null; + var height = null; + var range = null; + + // Save the height. Do this only whenever the window is resized so we don't + // recalculate often. + $(window).on('resize', function() { + height = $parent.height(); + range = $(document).height(); + }); + + // Find the current active section every scroll tick. + $parent.on('scroll', function() { + var y = $parent.scrollTop(); + y += height * (0.3 + 0.7 * Math.pow(y/range, 2)); + + var latest = null; + + for (var i in offsets) { + if (offsets.hasOwnProperty(i)) { + var offset = offsets[i]; + if ($(offset.el).offset().top + offset.offset < y) latest = offset; + } + } + + if (latest && (!current || (latest.index !== current.index))) { + callback.call($sections, + latest ? latest.id : null, + current ? current.id : null, + latest ? latest.el : null, + current ? current.el : null); + current = latest; + } + }); + + $(window).trigger('resize'); + $parent.trigger('scroll'); + + return this; + }; + +})(jQuery); +/*! Anchorjump (c) 2012, Rico Sta. Cruz. MIT License. + * http://github.com/rstacruz/jquery-stuff/tree/master/anchorjump */ + +// Makes anchor jumps happen with smooth scrolling. +// +// $("#menu a").anchorjump(); +// $("#menu a").anchorjump({ offset: -30 }); +// +// // Via delegate: +// $("#menu").anchorjump({ for: 'a', offset: -30 }); +// +// You may specify a parent. This makes it scroll down to the parent. +// Great for tabbed views. +// +// $('#menu a').anchorjump({ parent: '.anchor' }); +// +// You can jump to a given area. +// +// $.anchorjump('#bank-deposit', options); + +(function($) { + var defaults = { + 'speed': 500, + 'offset': 0, + 'for': null, + 'parent': null + }; + + $.fn.anchorjump = function(options) { + options = $.extend({}, defaults, options); + + if (options['for']) { + this.on('click', options['for'], onClick); + } else { + this.on('click', onClick); + } + + function onClick(e) { + var $a = $(e.target).closest('a'); + if (e.ctrlKey || e.metaKey || e.altKey || $a.attr('target')) return; + + e.preventDefault(); + var href = $a.attr('href'); + + $.anchorjump(href, options); + } + }; + + // Jump to a given area. + $.anchorjump = function(href, options) { + options = $.extend({}, defaults, options); + + var top = 0; + + if (href != '#') { + var $area = $(href); + // Find the parent + if (options.parent) { + var $parent = $area.closest(options.parent); + if ($parent.length) { $area = $parent; } + } + if (!$area.length) { return; } + + // Determine the pixel offset; use the default if not available + var offset = + $area.attr('data-anchor-offset') ? + parseInt($area.attr('data-anchor-offset'), 10) : + options.offset; + + top = Math.max(0, $area.offset().top + offset); + } + + $('html, body').animate({ scrollTop: top }, options.speed); + $('body').trigger('anchor', href); + + // Add the location hash via pushState. + if (window.history.pushState) { + window.history.pushState({ href: href }, "", href); + } + }; +})(jQuery); diff --git a/docs/style.css b/docs/style.css new file mode 100644 index 0000000..363856a --- /dev/null +++ b/docs/style.css @@ -0,0 +1,878 @@ +/* + +Please don't edit this file directly. +Instead, edit the stylus (.styl) files and compile it to CSS on your machine. + +*/ +/* ---------------------------------------------------------------------------- + * Fonts + */ +@import url("//fonts.googleapis.com/css?family=Montserrat:700|Open+Sans:300"); +/* ---------------------------------------------------------------------------- + * Base + */ +html, +body, +div, +span, +applet, +object, +iframe, +h1, +h2, +h3, +h4, +h5, +h6, +p, +blockquote, +pre, +a, +abbr, +acronym, +address, +big, +cite, +code, +del, +dfn, +em, +img, +ins, +kbd, +q, +s, +samp, +small, +strike, +strong, +sub, +sup, +tt, +var, +dl, +dt, +dd, +ol, +ul, +li, +fieldset, +form, +label, +legend, +table, +caption, +tbody, +tfoot, +thead, +tr, +th, +td { + margin: 0; + padding: 0; + border: 0; + outline: 0; + font-weight: inherit; + font-style: inherit; + font-family: inherit; + font-size: 100%; + vertical-align: baseline; +} +body { + line-height: 1; + color: #000; + background: #fff; +} +ol, +ul { + list-style: none; +} +table { + border-collapse: separate; + border-spacing: 0; + vertical-align: middle; +} +caption, +th, +td { + text-align: left; + font-weight: normal; + vertical-align: middle; +} +a img { + border: none; +} +html, +body { + height: 100%; +} +html { + overflow-x: hidden; +} +body, +td, +textarea, +input { + font-family: Helvetica Neue, Open Sans, sans-serif; + line-height: 1.6; + font-size: 13px; + color: #505050; +} +@media (max-width: 480px) { + body, + td, + textarea, + input { + font-size: 12px; + } +} +a { + color: #2badad; + text-decoration: none; +} +a:hover { + color: #228a8a; +} +/* ---------------------------------------------------------------------------- + * Content styling + */ +.content p, +.content ul, +.content ol, +.content h1, +.content h2, +.content h3, +.content h4, +.content h5, +.content h6, +.content pre, +.content blockquote { + padding: 10px 0; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; +} +.content h1, +.content h2, +.content h3, +.content h4, +.content h5, +.content h6 { + font-weight: bold; + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; +} +.content pre { + font-family: Menlo, monospace; +} +.content ul > li { + list-style-type: disc; +} +.content ol > li { + list-style-type: decimal; +} +.content ul, +.content ol { + margin-left: 20px; +} +.content ul > li { + list-style-type: none; + position: relative; +} +.content ul > li:before { + content: ''; + display: block; + position: absolute; + left: -17px; + top: 7px; + width: 5px; + height: 5px; + -webkit-border-radius: 4px; + border-radius: 4px; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; + background: #fff; + border: solid 1px #9090aa; +} +.content li > :first-child { + padding-top: 0; +} +.content strong, +.content b { + font-weight: bold; +} +.content i, +.content em { + font-style: italic; + color: #9090aa; +} +.content code { + font-family: Menlo, monospace; + background: #f3f6fb; + padding: 1px 3px; + font-size: 0.95em; +} +.content pre > code { + display: block; + background: transparent; + font-size: 0.85em; + letter-spacing: -1px; +} +.content blockquote :first-child { + padding-top: 0; +} +.content blockquote :last-child { + padding-bottom: 0; +} +.content table { + margin-top: 10px; + margin-bottom: 10px; + padding: 0; + border-collapse: collapse; + clear: both; + float: left; +} +.content table tr { + border-top: 1px solid #ccc; + background-color: #fff; + margin: 0; + padding: 0; +} +.content table tr :nth-child(2n) { + background-color: #f8f8f8; +} +.content table tr th { + text-align: auto; + font-weight: bold; + border: 1px solid #ccc; + margin: 0; + padding: 6px 13px; +} +.content table tr td { + text-align: auto; + border: 1px solid #ccc; + margin: 0; + padding: 6px 13px; +} +.content table tr th :first-child, +.content table tr td :first-child { + margin-top: 0; +} +.content table tr th :last-child, +.content table tr td :last-child { + margin-bottom: 0; +} +/* ---------------------------------------------------------------------------- + * Content + */ +.content-root { + min-height: 90%; + position: relative; +} +.content { + padding-top: 30px; + padding-bottom: 40px; + padding-left: 40px; + padding-right: 40px; + zoom: 1; + max-width: 700px; +} +.content:before, +.content:after { + content: ""; + display: table; +} +.content:after { + clear: both; +} +.content blockquote { + color: #9090aa; + text-shadow: 0 1px 0 rgba(255,255,255,0.5); +} +.content h1, +.content h2, +.content h3 { + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; + font-family: montserrat; + padding-bottom: 0; +} +.content h1 + p, +.content h2 + p, +.content h3 + p, +.content h1 ul, +.content h2 ul, +.content h3 ul, +.content h1 ol, +.content h2 ol, +.content h3 ol { + padding-top: 10px; +} +.content h1, +.content h2 { + text-transform: uppercase; + letter-spacing: 1px; + font-size: 1.5em; +} +.content h3 { + font-size: 1.2em; +} +.content h1, +.content h2, +.content .big-heading, +body.big-h3 .content h3 { + padding-top: 80px; +} +.content h1:before, +.content h2:before, +.content .big-heading:before, +body.big-h3 .content h3:before { + display: block; + content: ''; + background: -webkit-linear-gradient(left, #dfe2e7 80%, rgba(223,226,231,0)); + background: -moz-linear-gradient(left, #dfe2e7 80%, rgba(223,226,231,0)); + background: -o-linear-gradient(left, #dfe2e7 80%, rgba(223,226,231,0)); + background: -ms-linear-gradient(left, #dfe2e7 80%, rgba(223,226,231,0)); + background: linear-gradient(to right, #dfe2e7 80%, rgba(223,226,231,0)); + -webkit-box-shadow: 0 1px 0 rgba(255,255,255,0.4); + box-shadow: 0 1px 0 rgba(255,255,255,0.4); + height: 1px; + position: relative; + top: -40px; + left: -40px; + width: 100%; +} +@media (max-width: 768px) { + .content h1, + .content h2, + .content .big-heading, +body.big-h3 .content h3 { + padding-top: 40px; + } + .content h1:before, + .content h2:before, + .content .big-heading:before, +body.big-h3 .content h3:before { + background: #dfe2e7; + left: -40px; + top: -20px; + width: 120%; + } +} +.content h4, +.content h5, +.content .small-heading, +body:not(.big-h3) .content h3 { + border-bottom: solid 1px rgba(0,0,0,0.07); + color: #9090aa; + padding-top: 30px; + padding-bottom: 10px; +} +body:not(.big-h3) .content h3 { + font-size: 0.9em; +} +.content h1:first-child { + padding-top: 0; +} +.content h1:first-child, +.content h1:first-child a, +.content h1:first-child a:visited { + color: #505050; +} +.content h1:first-child:before { + display: none; +} +@media (max-width: 768px) { + .content h4, + .content h5, + .content .small-heading, + body:not(.big-h3) .content h3 { + padding-top: 20px; + } +} +@media (max-width: 480px) { + .content { + padding: 20px; + padding-top: 40px; + } + .content h4, + .content h5, + .content .small-heading, + body:not(.big-h3) .content h3 { + padding-top: 10px; + } +} +body.no-literate .content pre > code { + background: #f3f6fb; + border: solid 1px #e7eaee; + border-top: solid 1px #dbdde2; + border-left: solid 1px #e2e5e9; + display: block; + padding: 10px; + -webkit-border-radius: 2px; + border-radius: 2px; + overflow: auto; +} +body.no-literate .content pre > code { + -webkit-overflow-scrolling: touch; +} +body.no-literate .content pre > code::-webkit-scrollbar { + width: 15px; + height: 15px; +} +body.no-literate .content pre > code::-webkit-scrollbar-thumb { + background: #ddd; + -webkit-border-radius: 8px; + border-radius: 8px; + border: solid 4px #f3f6fb; +} +body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { + background: #999; + -webkit-box-shadow: inset 2px 2px 3px rgba(0,0,0,0.2); + box-shadow: inset 2px 2px 3px rgba(0,0,0,0.2); +} +@media (max-width: 1180px) { + .content pre > code { + background: #f3f6fb; + border: solid 1px #e7eaee; + border-top: solid 1px #dbdde2; + border-left: solid 1px #e2e5e9; + display: block; + padding: 10px; + -webkit-border-radius: 2px; + border-radius: 2px; + overflow: auto; + } + .content pre > code { + -webkit-overflow-scrolling: touch; + } + .content pre > code::-webkit-scrollbar { + width: 15px; + height: 15px; + } + .content pre > code::-webkit-scrollbar-thumb { + background: #ddd; + -webkit-border-radius: 8px; + border-radius: 8px; + border: solid 4px #f3f6fb; + } + .content pre > code:hover::-webkit-scrollbar-thumb { + background: #999; + -webkit-box-shadow: inset 2px 2px 3px rgba(0,0,0,0.2); + box-shadow: inset 2px 2px 3px rgba(0,0,0,0.2); + } +} +.button { + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; + font-family: montserrat, sans-serif; + letter-spacing: -1px; + font-weight: bold; + display: inline-block; + padding: 3px 25px; + border: solid 2px #2badad; + -webkit-border-radius: 20px; + border-radius: 20px; + margin-right: 15px; +} +.button, +.button:visited { + background: #2badad; + color: #fff; + text-shadow: none; +} +.button:hover { + border-color: #111; + background: #111; + color: #fff; +} +.button.light, +.button.light:visited { + background: transparent; + color: #9090aa; + border-color: #9090aa; + text-shadow: none; +} +.button.light:hover { + border-color: #9090aa; + background: #9090aa; + color: #fff; +} +.content .button + em { + color: #9090aa; +} +@media (min-width: 1180px) { + body:not(.no-literate) .content-root { + background-color: #f3f6fb; + -webkit-box-shadow: inset 780px 0 #fff, inset 781px 0 #dfe2e7, inset 790px 0 5px -10px rgba(0,0,0,0.1); + box-shadow: inset 780px 0 #fff, inset 781px 0 #dfe2e7, inset 790px 0 5px -10px rgba(0,0,0,0.1); + } +} +@media (min-width: 1180px) { + body:not(.no-literate) .content { + padding-left: 0; + padding-right: 0; + width: 930px; + max-width: none; + } + body:not(.no-literate) .content > p, + body:not(.no-literate) .content > ul, + body:not(.no-literate) .content > ol, + body:not(.no-literate) .content > h1, + body:not(.no-literate) .content > h2, + body:not(.no-literate) .content > h3, + body:not(.no-literate) .content > h4, + body:not(.no-literate) .content > h5, + body:not(.no-literate) .content > h6, + body:not(.no-literate) .content > pre, + body:not(.no-literate) .content > blockquote { + width: 550px; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; + padding-right: 40px; + padding-left: 40px; + } + body:not(.no-literate) .content > h1, + body:not(.no-literate) .content > h2, + body:not(.no-literate) .content > h3 { + clear: both; + width: 100%; + } + body:not(.no-literate) .content > pre, + body:not(.no-literate) .content > blockquote { + width: 380px; + padding-left: 20px; + padding-right: 20px; + float: right; + clear: right; + } + body:not(.no-literate) .content > pre + p, + body:not(.no-literate) .content > blockquote + p, + body:not(.no-literate) .content > pre + ul, + body:not(.no-literate) .content > blockquote + ul, + body:not(.no-literate) .content > pre + ol, + body:not(.no-literate) .content > blockquote + ol, + body:not(.no-literate) .content > pre + h4, + body:not(.no-literate) .content > blockquote + h4, + body:not(.no-literate) .content > pre + h5, + body:not(.no-literate) .content > blockquote + h5, + body:not(.no-literate) .content > pre + h6, + body:not(.no-literate) .content > blockquote + h6 { + clear: both; + } + body:not(.no-literate) .content > p, + body:not(.no-literate) .content > ul, + body:not(.no-literate) .content > ol, + body:not(.no-literate) .content > h4, + body:not(.no-literate) .content > h5, + body:not(.no-literate) .content > h6 { + float: left; + clear: left; + } + body:not(.no-literate) .content > h4, + body:not(.no-literate) .content > h5, + body:not(.no-literate) .content > .small-heading, + body:not(.big-h3) body:not(.no-literate) .content > h3 { + margin-left: 40px; + width: 470px; + margin-bottom: 3px; + padding-left: 0; + padding-right: 0; + } + body:not(.no-literate) .content > table { + margin-left: 40px; + margin-right: 40px; + max-width: 470px; + } + body:not(.no-literate):not(.big-h3) .content > h3 { + margin-left: 40px; + width: 470px; + margin-bottom: 3px; + padding-left: 0; + padding-right: 0; + } +} +.header { + background: #f3f6fb; + text-shadow: 0 1px 0 rgba(255,255,255,0.5); + border-bottom: solid 1px #dfe2e7; + padding: 15px 15px 15px 30px; + zoom: 1; + line-height: 20px; + position: relative; +} +.header:before, +.header:after { + content: ""; + display: table; +} +.header:after { + clear: both; +} +.header .left { + float: left; +} +.header .right { + text-align: right; + position: absolute; + right: 15px; + top: 15px; +} +.header .right iframe { + display: inline-block; + vertical-align: middle; +} +.header h1 { + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; + font-weight: bold; + font-family: montserrat, sans-serif; + font-size: 13px; +} +.header h1, +.header h1 a, +.header h1 a:visited { + color: #9090aa; +} +.header h1 a:hover { + color: #505050; +} +.header li a { + font-size: 0.88em; + color: #9090aa; + display: block; +} +.header li a:hover { + color: #3a3a44; +} +@media (min-width: 480px) { + .header h1 { + float: left; + } + .header ul, + .header li { + display: block; + float: left; + } + .header ul { + margin-left: -15px; + } + .header h1 + ul { + border-left: solid 1px #dfe2e7; + margin-left: 15px; + } + .header li { + border-left: solid 1px rgba(255,255,255,0.5); + border-right: solid 1px #dfe2e7; + } + .header li:last-child { + border-right: 0; + } + .header li a { + padding: 0 15px; + } +} +@media (max-width: 480px) { + .right { + display: none; + } +} +.menubar { + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; +} +.menubar .section { + padding: 30px 30px; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; +} +.menubar .section + .section { + border-top: solid 1px #dfe2e7; +} +.menubar .section.no-line { + border-top: 0; + padding-top: 0; +} +a.big.button { + display: block; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; + width: 100%; + padding: 10px 20px; + text-align: center; + font-weight: bold; + font-size: 1.1em; + background: transparent; + border: solid 3px #2badad; + -webkit-border-radius: 30px; + border-radius: 30px; + font-family: montserrat, sans-serif; +} +a.big.button, +a.big.button:visited { + color: #2badad; + text-decoration: none; +} +a.big.button:hover { + background: #2badad; +} +a.big.button:hover, +a.big.button:hover:visited { + color: #fff; +} +@media (max-width: 480px) { + .menubar { + padding: 20px; + border-bottom: solid 1px #dfe2e7; + } +} +@media (max-width: 768px) { + .menubar { + display: none; + } +} +@media (min-width: 768px) { + .content-root { + padding-left: 230px; + } + .menubar { + position: absolute; + left: 0; + top: 0; + bottom: 0; + width: 230px; + border-right: solid 1px #dfe2e7; + } + .menubar.fixed { + position: fixed; + overflow-y: auto; + } + .menubar.fixed { + -webkit-overflow-scrolling: touch; + } + .menubar.fixed::-webkit-scrollbar { + width: 15px; + height: 15px; + } + .menubar.fixed::-webkit-scrollbar-thumb { + background: #ddd; + -webkit-border-radius: 8px; + border-radius: 8px; + border: solid 4px #fff; + } + .menubar.fixed:hover::-webkit-scrollbar-thumb { + background: #999; + -webkit-box-shadow: inset 2px 2px 3px rgba(0,0,0,0.2); + box-shadow: inset 2px 2px 3px rgba(0,0,0,0.2); + } +} +.menubar { + font-size: 0.9em; +} +.menu ul.level-1 > li + li { + margin-top: 20px; +} +.menu a { + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; + position: relative; + display: block; + padding-top: 1px; + padding-bottom: 1px; + margin-right: -30px; +} +.menu a, +.menu a:visited { + color: #2badad; +} +.menu a:hover { + color: #228a8a; +} +.menu a.level-1 { + font-family: montserrat, sans-serif; + text-transform: uppercase; + font-size: 0.9em; + font-weight: bold; +} +.menu a.level-1, +.menu a.level-1:visited { + color: #9090aa; +} +.menu a.level-1:hover { + color: #565666; +} +.menu a.level-2 { + font-weight: normal; +} +.menu a.level-3 { + font-weight: normal; + font-size: 0.9em; + padding-left: 10px; +} +.menu a.active { + font-weight: bold !important; +} +.menu a.active, +.menu a.active:visited, +.menu a.active:hover { + color: #505050 !important; +} +.menu a.active:after { + content: ''; + display: block; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; + position: absolute; + top: 10px; + right: 30px; + width: 9px; + height: 3px; + -webkit-border-radius: 2px; + border-radius: 2px; + background: #2badad; +} +code .string, +code .number { + color: #3ac; +} +code .init { + color: #383; +} +code .keyword { + font-weight: bold; +} +code .comment { + color: #adadcc; +} +.large-brief .content > h1:first-child + p, +.content > p.brief { + font-size: 1.3em; + font-family: Open Sans, sans-serif; + font-weight: 300; +} +.title-area { + min-height: 100px; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; + text-align: center; + border-bottom: solid 1px #dfe2e7; + overflow: hidden; +} +.title-area > img.bg { + z-index: 0; + position: absolute; + left: -9999px; +} +.title-area > div { + position: relative; + z-index: 1; +} From 4b1993e1b3ada1e885e0003185070a12e43ccd2f Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:00:35 -0500 Subject: [PATCH 111/150] fix colors --- docs/style.css | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/style.css b/docs/style.css index 363856a..b22dfcf 100644 --- a/docs/style.css +++ b/docs/style.css @@ -116,7 +116,7 @@ input { font-family: Helvetica Neue, Open Sans, sans-serif; line-height: 1.6; font-size: 13px; - color: #505050; + color: #454545; } @media (max-width: 480px) { body, @@ -378,7 +378,7 @@ body:not(.big-h3) .content h3 { .content h1:first-child, .content h1:first-child a, .content h1:first-child a:visited { - color: #505050; + color: #0076de; } .content h1:first-child:before { display: none; @@ -637,7 +637,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { color: #9090aa; } .header h1 a:hover { - color: #505050; + color: #0076de; } .header li a { font-size: 0.88em; @@ -820,7 +820,7 @@ a.big.button:hover:visited { .menu a.active, .menu a.active:visited, .menu a.active:hover { - color: #505050 !important; + color: #0076de !important; } .menu a.active:after { content: ''; From 13496198e980cf1a29f715201ce18ed430c4c119 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:04:11 -0500 Subject: [PATCH 112/150] fix colors --- docs/style.css | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/style.css b/docs/style.css index b22dfcf..181b08d 100644 --- a/docs/style.css +++ b/docs/style.css @@ -81,7 +81,7 @@ td { body { line-height: 1; color: #000; - background: #fff; + background: #f6f6f6; } ol, ul { @@ -127,7 +127,7 @@ input { } } a { - color: #2badad; + color: #454545; text-decoration: none; } a:hover { @@ -471,14 +471,14 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { font-weight: bold; display: inline-block; padding: 3px 25px; - border: solid 2px #2badad; + border: solid 2px #454545; -webkit-border-radius: 20px; border-radius: 20px; margin-right: 15px; } .button, .button:visited { - background: #2badad; + background: #454545; color: #fff; text-shadow: none; } @@ -707,18 +707,18 @@ a.big.button { font-weight: bold; font-size: 1.1em; background: transparent; - border: solid 3px #2badad; + border: solid 3px #454545; -webkit-border-radius: 30px; border-radius: 30px; font-family: montserrat, sans-serif; } a.big.button, a.big.button:visited { - color: #2badad; + color: #454545; text-decoration: none; } a.big.button:hover { - background: #2badad; + background: #454545; } a.big.button:hover, a.big.button:hover:visited { @@ -788,7 +788,7 @@ a.big.button:hover:visited { } .menu a, .menu a:visited { - color: #2badad; + color: #454545; } .menu a:hover { color: #228a8a; @@ -835,7 +835,7 @@ a.big.button:hover:visited { height: 3px; -webkit-border-radius: 2px; border-radius: 2px; - background: #2badad; + background: #454545; } code .string, code .number { From 88efd6586187ee8675af9aaece36bda260204b06 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:12:30 -0500 Subject: [PATCH 113/150] use Lato font --- docs/style.css | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/style.css b/docs/style.css index 181b08d..299bbe5 100644 --- a/docs/style.css +++ b/docs/style.css @@ -74,7 +74,7 @@ td { outline: 0; font-weight: inherit; font-style: inherit; - font-family: inherit; + font-family: Lato; font-size: 100%; vertical-align: baseline; } @@ -113,10 +113,10 @@ body, td, textarea, input { - font-family: Helvetica Neue, Open Sans, sans-serif; + font-family: Lato; line-height: 1.6; font-size: 13px; - color: #454545; + color: black; } @media (max-width: 480px) { body, @@ -163,7 +163,7 @@ a:hover { text-rendering: optimizeLegibility; } .content pre { - font-family: Menlo, monospace; + font-family: Lato; } .content ul > li { list-style-type: disc; @@ -295,7 +295,7 @@ a:hover { .content h3 { -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; - font-family: montserrat; + font-family: Lato; padding-bottom: 0; } .content h1 + p, @@ -466,7 +466,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { .button { -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; - font-family: montserrat, sans-serif; + font-family: Lato; letter-spacing: -1px; font-weight: bold; display: inline-block; @@ -595,7 +595,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { } } .header { - background: #f3f6fb; + background: white; text-shadow: 0 1px 0 rgba(255,255,255,0.5); border-bottom: solid 1px #dfe2e7; padding: 15px 15px 15px 30px; @@ -628,7 +628,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; font-weight: bold; - font-family: montserrat, sans-serif; + font-family: Lato; font-size: 13px; } .header h1, @@ -645,7 +645,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { display: block; } .header li a:hover { - color: #3a3a44; + color: #0076de; } @media (min-width: 480px) { .header h1 { @@ -710,7 +710,7 @@ a.big.button { border: solid 3px #454545; -webkit-border-radius: 30px; border-radius: 30px; - font-family: montserrat, sans-serif; + font-family: Lato; } a.big.button, a.big.button:visited { @@ -794,7 +794,7 @@ a.big.button:hover:visited { color: #228a8a; } .menu a.level-1 { - font-family: montserrat, sans-serif; + font-family: Lato; text-transform: uppercase; font-size: 0.9em; font-weight: bold; @@ -853,7 +853,7 @@ code .comment { .large-brief .content > h1:first-child + p, .content > p.brief { font-size: 1.3em; - font-family: Open Sans, sans-serif; + font-family: Lato; font-weight: 300; } .title-area { From 6ccae5fa06c46bd6ebf5770dff9b8673f2e0456d Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:14:15 -0500 Subject: [PATCH 114/150] add lato font tag --- docs/index.html | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/index.html b/docs/index.html index a08010b..d04e7f3 100644 --- a/docs/index.html +++ b/docs/index.html @@ -18,6 +18,8 @@ + + Contexture From 9703f779ba82893f84d738f57c67b47f83abd5a5 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:16:55 -0500 Subject: [PATCH 115/150] fix hover colors --- docs/style.css | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/style.css b/docs/style.css index 299bbe5..d9ee9b9 100644 --- a/docs/style.css +++ b/docs/style.css @@ -131,7 +131,7 @@ a { text-decoration: none; } a:hover { - color: #228a8a; + color: #0076de; } /* ---------------------------------------------------------------------------- * Content styling @@ -791,7 +791,7 @@ a.big.button:hover:visited { color: #454545; } .menu a:hover { - color: #228a8a; + color: #0076de; } .menu a.level-1 { font-family: Lato; @@ -804,7 +804,7 @@ a.big.button:hover:visited { color: #9090aa; } .menu a.level-1:hover { - color: #565666; + color: #0076de; } .menu a.level-2 { font-weight: normal; From 0c6fd882d915a3ea611e2bd5f9ac1c5006c7de52 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:21:52 -0500 Subject: [PATCH 116/150] fix docs formatting --- README.md | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 1f85805..c3c7f46 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,7 @@ been doing for almost a decade. #### Contexture > NOUN + > 1. The fact or manner of being woven or linked together to form a > connected whole. > 1.1. A mass of things interwoven together; a fabric. @@ -151,10 +152,7 @@ different search representations, about the databases involved, and the DSL, then outputs the search results respective to each one of the queries described in a copy of the received DSL. -You can read more about the core here: -- [In the repository](https://github.com/smartprocure/contexture). -- In our [docs about querying (Contexture Core section)](../querying/contexture-core.md). -- Or with greater detail in our [under the hood docs (Contexture Core section)](../under-the-hood/contexture-core.md). +You can read more about the core here, [in the repository](https://github.com/smartprocure/contexture). #### Contexture Providers @@ -162,9 +160,8 @@ The Contexture Providers are the interfacing layer that ties the Contexture DSL to the targeted databases. So far, we have only two open source providers: -- [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch), - and -- [contexture-mongo](https://github.com/smartprocure/contexture-mongo). +- [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) +- [contexture-mongo](https://github.com/smartprocure/contexture-mongo) If you are planning to use either ElasticSearch or MongoDB for your project, the best way to get started is to use those repositories. @@ -173,10 +170,6 @@ need to implement the provider yourself. We're looking for code contributors, so please don't feel limited to the current available tools. Help us grow together! -You can read more about Contexture Providers here: -- In our [docs about querying (Available Providers section)](../querying/available-providers.md). -- In with greater detail in our [under the hood docs (Contexture Providers section)](../under-the-hood/contexture-providers/README.md). - #### Contexture Client The Contexture Client is responsible for triggering behaviors on the @@ -184,10 +177,7 @@ search interfaces by knowing what causes changes in one or more elements of the search tree. It is the key piece of technology that allows our search interfaces to work in real time. -You can read more about the Contexture Client here: -- [In the repository](https://github.com/smartprocure/contexture-client). -- In our [docs about querying (Contexture Client section)](../interactive-queries/contexture-client.md). -- Or with greater detail in our [under the hood docs (Contexture Client section)](../under-the-hood/contexture-client.md). +You can read more about the Contexture Client here, [in the repository](https://github.com/smartprocure/contexture-client). #### Contexture React @@ -195,11 +185,7 @@ The Contexture React repository holds a list of components that facilitate building search interfaces. They are mainly graphical representations of the types that exist on our Contexture Providers. -You can read more about the Contexture React: -- [In the repository](https://github.com/smartprocure/contexture-client). -- In our guide for the [Available React Components for Types](./types/react-components.md). -- Or in greater detail in our [under the hood docs (Contexture React section)](under-the-hood/contexture-react.md). - +You can read more about Contexture React here, [in the repository](https://github.com/smartprocure/contexture-client). ### Brief History **TODO** From d148ce9b34caec9a943a6742b217b6b450075d25 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:37:20 -0500 Subject: [PATCH 117/150] add first half of getting started section --- README.md | 522 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 521 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c3c7f46..d4fceaa 100644 --- a/README.md +++ b/README.md @@ -194,17 +194,537 @@ You can read more about Contexture React here, [in the repository](https://githu **TODO** ## Getting Started -### Project Setup #### Install Node 9 and NPM +NodeJS 9 and NPM can be installed through [the list of previous +releases of NodeJS](https://nodejs.org/en/download/releases/). You +might get it working with Node 10 (if so, let us know). We haven't +fully upgraded to Node 10 yet, so until then, we encourage you to at +least have a working version of Node 9 in hand. + +An easy way to move from one version to another is with +[nvm](https://github.com/creationix/nvm). Here's a command you can run +to install nvm: + + `curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash` + +Please follow [nvm's README](https://github.com/creationix/nvm/blob/master/README.md) for more information. + #### Install Contexture +Once you have NodeJS and NPM installed, you'll need either a new +folder for your new project with Contexture, or to go to an existing +project, then run: + + `npm install --save contexture` + #### Install Contexture Client +To install the `contexture-client` you can run the following +command in your project's root folder: + + `npm install --save contexture-client` + #### Install Contexture ElasticSearch +To install the `contexture-elasticsearch` you can also run the +following command in your project's root folder: + + `npm install --save contexture-elasticsearch` + #### Install Contexture Mongo +To install the `contexture-mongo` you can also run the following +command in your project's root folder: + + `npm install --save contexture-mongo` + #### Install Contexture React +To install the `contexture-react` you can also run the following +command in your project's root folder: + + `npm install --save contexture-react` + ### A First Contexture Script +With everything installed, let's see a simple script that runs a +simple one-time search: + +```javascript +let contexture = require('contexture') + +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} + +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} + +let result = await contexture({ + schemas, + providers: { + mongo: require('contexture-mongo')({ + types: require('contexture-mongo/types')(), + getClient: () => ({ + // Fake mongo client. + // For this example we only care about + // collection().aggregate([...]).toArray() being a promise. + collection: name => ({ + aggregate: aggregations => ({ + toArray: async () => ['Unrealistic result example'] + }) + }) + }) + }) + }, +}, searchTree) + +console.log(result.children[1].context.response.results) + +// Explore it yourself! +result +``` + +You can also try this same code in Runkit here: + + +## What it does + +1. Requiring Contexture + +```javascript +let contexture = require('contexture') +``` +We start by requiring `contexture`. There's nothing much else to see +here. + +![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) + +2. Writing the Schemas + +```javascript +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} +``` +Immediatly afterwards, we define our schemas. For this specific +example, we are going to emulate doing a search on a single collection +of a Mongo database. We define `collectionName` as the name of this +arbitrary collection. The `schemas` object ends up containing a single +schema with a key being `collectionNameSchema`, which is going to be +supported by a single provider `mongo`, from which we'll be looking +for the `collectionName` collection. + +3. Writing the Contexture DSL + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} +``` +Our next step is to define a simple search query using Contexture DSL. +This search query will have a mandatory root group, which among other +features indicates which schema we will be running this query on +(`collectionNameSchema`). This node will have two children. The first +children is going to be our search query. This query will be a `text` +type query. We're indicating that we want to run a plain text search +that will try to match any record which `name` contains the word +`value`. The next node has the `results` type. This is where the +results will be written once the search runs. + +3. Getting Results + +```javascript +let result = await contexture({ + schemas, + providers: { + mongo: require('contexture-mongo')({ + types: require('contexture-mongo/types')(), + getClient: () => ({ + // Fake mongo client. + // For this example we only care about + // collection().aggregate([...]).toArray() being a promise. + collection: name => ({ + aggregate: aggregations => ({ + toArray: async () => ['Unrealistic result example'] + }) + }) + }) + }) + }, +}, searchTree) +``` + +The next thing we do is to actually run a one-time search. In this +case, we `await` for `contexture`, passing along some very important +parameters. First, we pass the schemas we previously defined. Then, we +pass a providers object, which has the provider for `mongo`. This +`mongo` key will be matched against the `mongo` key that we defined in +the schemas, so you could change this property name to something +entirely different. When we send the mongo provider, we assign the +result of the initialization of `contexture-mongo`, where we send the +`contxture-mongo/types` (which **needs to be called as a function once +required**) and a `getClient` function. In a real schenario, you would +just send the object that results of `require('mongodb')`. However, to +provide an exceutable example in the browser, we've made a very small +Mock of MongoDB where we will return a same object for any collection +call, which will only allow fake aggregations, which will have a +promise `toArray` function that will return our `Unrealistic result +example`. + +Keep in mind that since `contexture` returns a promise, you can change +`await contexture({ /* .. */ })` to `contexture({ /* .. */ }).then()`, +but you'll need to move the code that we have after the await call +into the function that is passed on the `.then()` call. + +4. Exploring the Results + +```javascript +console.log(result.children[1].context.response.results) + +// Explore it yourself! +result +``` + +Finally, we can use our search result! The output of the search will +be added to a copy of the original search query. The location of this +result will be in the children of the `root` node that has the +`results` type, within a `context` object, within a `response` object, +on a `results` property. This whole object is going to be exposed in +the _runkit_ example for you to play with it. + ### Connecting to Elasticsearch & MongoDB +Our primary use case for advanced search interfaces is to query data +that we store on ElasticSearch and MongoDB. Because of that, we +provide two contexture libraries for those databases. In this page +we'll examine how to use these repositories to connect to existing +Mongo databases and ElasticSearch indexes. + #### Connecting to ElasticSearch +The followng code example shows how to connect to ElasticSearch: + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-elasticsearch') +let types = require('contexture-elasticsearch/types') +let elasticsearch = require('elasticsearch') +let AgentKeepAlive = require('agentkeepalive') + +let elasticClient = null +let getClient = () => { + if (elasticClient) return elasticClient + elasticClient = elasticsearch.Client({ + minSockets: 1, + maxSockets: 20, + keepAlive: true, + createNodeAgent: (connection, config) => + new AgentKeepAlive(connection.makeAgentConfig(config)) + }) + return elasticClient +} + +let schemas = { + yourCustomSchemaName: { + elasticsearch: { + index: 'SomeIndex', + type: 'SomeType' + } + } +} + +let search = Contexture({ + schemas, + providers: { + elasticsearch: provider({ + getClient, + request: { + headers: { + 'custom-header-app-name': 'my-app-sent-this' + } + }, + types: types() + }) + } +}) +``` + +The code above will provide a working search function `search` that +will transform any given query into a working ElasticSearch query, +which will be sent to the database to retrieve the data. Let's examine +this code in greater detail. + +1. The Dependencies + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-elasticsearch') +let types = require('contexture-elasticsearch/types') +let elasticsearch = require('elasticsearch') +let AgentKeepAlive = require('agentkeepalive') +``` + +The first six lines are about requiring dependencies, the first +dependency being just `contexture`. The second one is our database +provider, `contexture-elasticsearch`. Then, we require the types; even +though you will probably have to write your own types, we have some +pre-defined types available at `contexture-elasticsearch/types`. +Lastly, we require `elasticsearch` and `agentkeepalive`, so we can +actually connect to ElasticSearch and keep it connected even if the +address becomes unreachable for a while. + +Please feel free to dig around these topics by following these links: + +- [Contexture Providers](under-the-hood/contexture-providers/README.md). +- [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). +- [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). + +2. The ElasticSearch Client + +```javascript +let elasticClient = null +let getClient = () => { + if (elasticClient) return elasticClient + elasticClient = elasticsearch.Client({ + minSockets: 1, + maxSockets: 20, + keepAlive: true, + createNodeAgent: (connection, config) => + new AgentKeepAlive(connection.makeAgentConfig(config)) + }) + return elasticClient +} +``` + +Now, we define a utility function to connect to ElasticSearch just +once. The idea here is to be able to re-use the same connection +instead of using new clients every time a search is executed. We do +this by declaring a function that will return `elasticClient` if it has +a truthy value, or otherwise instantiate a new elasticsearch client +with the given configuration. The configuration we are sending is +merely an example and shouldn't be used without understanding what is +being expected by the elasticsearch library. More on the following +links: + +- [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). + +3. The Schemas + +```javascript +let schemas = { + elasticsearch: { + index: 'SomeIndex', + type: 'SomeType' + } +} +``` + +As shown above, the next thing we do is to define the schemas. This is +an object which properties are the names of each one of the schemas +that will be available for the contexture DSL. The schemas for the +ElasticSearch provider can specify any or all of the following +properties: + +| Option | Type | Description | Required | +| ------ | ---- | ----------- | -------- | +| `index` | `string` | Which ES index to use when querying | x | +| `type` | `string` | Which ES type to use when querying | | +| `summaryView` | `function` | Used by `results` to return a summary view instead of the whole document, (e.g. for indexes with many fields). Defaults to returning the `hit` property. | | +| `highlight` | `object` | Used by `results` to determine what fields to highlight, and whether or not they are `inline` (copied over inline on to the source) or `additional` (in a list of additional fields that matched) | | +| `forceExclude` | `array` | Used by `results` to extend the exclude fields provided on the search tree. The extension happens only if the results node has a `forceExclude` flag set to true. + +You can read more about these here: + +- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). + +4. Our Search Function + +```javascript +let search = Contexture({ + schemas, + providers: { + elasticsearch: provider({ + getClient, + request: { + headers: { + 'custom-header-app-name': 'my-app-sent-this' + } + }, + types: types() + }) + } +}) +``` + +Once we have the schemas set, we can create our `search` function. +For this purpose, we will be calling `Contexture` with just one +object. This object will have the `schemas`, and the `providers`. The +providers will host just one key/value, the one specific for +`elasticsearch`. The provider, which we required at the beginning of +the script, needs to be called with the `getClient` function we just +created and the `types()` that we got from the +`contexture-elasticsearch/types` repository. We also show that you can +customize the request headers by providing an object that includes the +headers keys and values. This `search` function is ready to receive +search trees and write the results back! + +This example and many other important details about +`contexture-elasticsearch` are accessible in the following links: + +- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). + #### Connecting to MongoDB +The followng code example shows how to connect to MongoDB: + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient + +let schemas = { + yourCustomSchemaName: { + mongo: { + collection: 'SomeCollection' + } + } +} + +let search = null + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +The code above will provide a working search function `search` that +will transform any given query into a working MongoDB query, +which will be sent to the database to retrieve the data. Let's examine +this code in greater detail. + +1. The Dependencies + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient +``` + +The first lines are about requiring dependencies, the first +dependency being just `contexture`. The second one is our database +provider, `contexture-mongo`. Then, we require the types; even +though you will probably have to write your own types, we have some +pre-defined types available at `contexture-mongo/types`. +Lastly, we require `mongodb` to get the much needed `MongoClient`, so +we can actually connect to the database. + +Please feel free to dig around these topics by following these links: + +- [Contexture Providers](under-the-hood/contexture-providers/README.md). +- [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). + +2. The Schemas + +```javascript +let schemas = { + yourCustomSchemaName: { + mongo: { + collection: 'SomeCollection' + } + } +} +``` + +As shown above, the next thing we do is to define the schemas. This is +an object which properties are the names of each one of the schemas +that will be available for the contexture DSL. The schemas for the +MongoDB provider can specify any or all of the following +properties: + +| Option | Type | Description | Required | +| ------ | ---- | ----------- | -------- | +| `collection` | `string` | The MongoDB collection that will be used to run the queries | x | + +You can read more about these here: + +- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). + +3. The MongoDB Client & Our Search Function + +```javascript +let search = null + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +Next we will need to connect to MongoDB. In the previous code, we +provide an example in which we connect to `mongodb://localhost:27017`, +and using a callback to the `connect` method, we are able to obtain +the database client that we need. Once we have this client, we can +actually create our `search` function. For this purpose, we will +be calling `Contexture` with just one object. This object will have +the `schemas`, and the `providers`. The providers will host just one +key/value, the one specific for `mongo`. The provider, which +we required at the beginning of the script, needs to be called with +a `getClient` function that will just return the database client, and +the `types()` that we got from the `contexture-mongo/types` repository. +With these steps completed, we end up with a function that is ready to +receive search trees and write the results back! + +You can read more about these topics in the following links: + +- [Connecting to MongoDB using the native MongoDB driver for NodeJS](http://mongodb.github.io/node-mongodb-native/api-articles/nodekoarticle1.html#getting-that-connection-to-the-database). + +This example and many other important details about +`contexture-mongo` are accessible in the following links: + +- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). + ### Connecting to Other Databases ### Simple Search Box ### Your First Filter From fda50cb25c95e9ca14db7fe53508e5e4d2d7715e Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:46:06 -0500 Subject: [PATCH 118/150] make font more readable --- docs/style.css | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/docs/style.css b/docs/style.css index d9ee9b9..ed8f307 100644 --- a/docs/style.css +++ b/docs/style.css @@ -75,7 +75,7 @@ td { font-weight: inherit; font-style: inherit; font-family: Lato; - font-size: 100%; + font-size: 1em; vertical-align: baseline; } body { @@ -822,21 +822,6 @@ a.big.button:hover:visited { .menu a.active:hover { color: #0076de !important; } -.menu a.active:after { - content: ''; - display: block; - -webkit-box-sizing: border-box; - -moz-box-sizing: border-box; - box-sizing: border-box; - position: absolute; - top: 10px; - right: 30px; - width: 9px; - height: 3px; - -webkit-border-radius: 2px; - border-radius: 2px; - background: #454545; -} code .string, code .number { color: #3ac; From d0591be0e3ede0237f8fc8cd2e1437a8270f3f64 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:48:04 -0500 Subject: [PATCH 119/150] change font used --- docs/style.css | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/style.css b/docs/style.css index ed8f307..978add9 100644 --- a/docs/style.css +++ b/docs/style.css @@ -74,7 +74,7 @@ td { outline: 0; font-weight: inherit; font-style: inherit; - font-family: Lato; + font-family: -apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,Arial,sans-serif,Apple Color Emoji,Segoe UI Emoji; font-size: 1em; vertical-align: baseline; } @@ -208,7 +208,7 @@ a:hover { color: #9090aa; } .content code { - font-family: Menlo, monospace; + font-family: SFMono-Regular,Consolas,Liberation Mono,Menlo,monospace; background: #f3f6fb; padding: 1px 3px; font-size: 0.95em; From 5620a86cd8dd26b8bac893c958e16bf4818a849f Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:49:43 -0500 Subject: [PATCH 120/150] change font size --- docs/style.css | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/style.css b/docs/style.css index 978add9..1509f38 100644 --- a/docs/style.css +++ b/docs/style.css @@ -75,7 +75,7 @@ td { font-weight: inherit; font-style: inherit; font-family: -apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,Arial,sans-serif,Apple Color Emoji,Segoe UI Emoji; - font-size: 1em; + font-size: 1.02em; vertical-align: baseline; } body { From 3704a8b439442fa34065657393efcd07de8b7b99 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 19:56:53 -0500 Subject: [PATCH 121/150] fix fonts --- docs/style.css | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/docs/style.css b/docs/style.css index 1509f38..520f474 100644 --- a/docs/style.css +++ b/docs/style.css @@ -33,7 +33,6 @@ acronym, address, big, cite, -code, del, dfn, em, @@ -113,7 +112,7 @@ body, td, textarea, input { - font-family: Lato; + font-family: -apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,Arial,sans-serif,Apple Color Emoji,Segoe UI Emoji; line-height: 1.6; font-size: 13px; color: black; @@ -163,7 +162,7 @@ a:hover { text-rendering: optimizeLegibility; } .content pre { - font-family: Lato; + font-family: inherit; } .content ul > li { list-style-type: disc; @@ -295,7 +294,7 @@ a:hover { .content h3 { -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; - font-family: Lato; + font-family: inherit; padding-bottom: 0; } .content h1 + p, @@ -466,7 +465,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { .button { -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; - font-family: Lato; + font-family: inherit; letter-spacing: -1px; font-weight: bold; display: inline-block; @@ -628,7 +627,7 @@ body.no-literate .content pre > code:hover::-webkit-scrollbar-thumb { -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; font-weight: bold; - font-family: Lato; + font-family: inherit; font-size: 13px; } .header h1, @@ -710,7 +709,7 @@ a.big.button { border: solid 3px #454545; -webkit-border-radius: 30px; border-radius: 30px; - font-family: Lato; + font-family: inherit; } a.big.button, a.big.button:visited { @@ -794,7 +793,7 @@ a.big.button:hover:visited { color: #0076de; } .menu a.level-1 { - font-family: Lato; + font-family: inherit; text-transform: uppercase; font-size: 0.9em; font-weight: bold; @@ -822,12 +821,15 @@ a.big.button:hover:visited { .menu a.active:hover { color: #0076de !important; } +code { + font-family: SFMono-Regular,Consolas,Liberation Mono,Menlo,monospace; +} code .string, code .number { - color: #3ac; + color: #032f62; } code .init { - color: #383; + color: #e36209; } code .keyword { font-weight: bold; @@ -838,7 +840,7 @@ code .comment { .large-brief .content > h1:first-child + p, .content > p.brief { font-size: 1.3em; - font-family: Lato; + font-family: inherit; font-weight: 300; } .title-area { From 39bc2148eccda7f7eaac92767bba28b95daaac04 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:00:03 -0500 Subject: [PATCH 122/150] fix code span styling --- docs/style.css | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/style.css b/docs/style.css index 520f474..b50db90 100644 --- a/docs/style.css +++ b/docs/style.css @@ -14,7 +14,6 @@ Instead, edit the stylus (.styl) files and compile it to CSS on your machine. html, body, div, -span, applet, object, iframe, @@ -821,7 +820,7 @@ a.big.button:hover:visited { .menu a.active:hover { color: #0076de !important; } -code { +span, code { font-family: SFMono-Regular,Consolas,Liberation Mono,Menlo,monospace; } code .string, From bac03c9a2449ece7c009935c22ac26722b5db19a Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:06:32 -0500 Subject: [PATCH 123/150] fix link color --- docs/style.css | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/style.css b/docs/style.css index b50db90..ad9a185 100644 --- a/docs/style.css +++ b/docs/style.css @@ -125,7 +125,7 @@ input { } } a { - color: #454545; + color: #0366d6; text-decoration: none; } a:hover { From 24564c2fe4e2e754bb8e72146e48d86d74eec774 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:13:10 -0500 Subject: [PATCH 124/150] add code colors --- docs/style.css | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/style.css b/docs/style.css index ad9a185..6f62759 100644 --- a/docs/style.css +++ b/docs/style.css @@ -822,6 +822,7 @@ a.big.button:hover:visited { } span, code { font-family: SFMono-Regular,Consolas,Liberation Mono,Menlo,monospace; + color: #528186; } code .string, code .number { @@ -836,6 +837,26 @@ code .keyword { code .comment { color: #adadcc; } +code .symbol { + color: #000; +} +code .type { + color: #6e45a5; +} + +.lang-swift .number { + color: #1f1bcb; +} + +.lang-swift .string, +.lang-swift .string * { + color: #c21d55; +} + +.lang-swift .comment, +.lang-swift .comment * { + color: #10821d; +} .large-brief .content > h1:first-child + p, .content > p.brief { font-size: 1.3em; From b263d9ec54488942786431ff12258efab7d8781e Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:17:54 -0500 Subject: [PATCH 125/150] fix code colors --- docs/style.css | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/style.css b/docs/style.css index 6f62759..a307b4b 100644 --- a/docs/style.css +++ b/docs/style.css @@ -822,7 +822,7 @@ a.big.button:hover:visited { } span, code { font-family: SFMono-Regular,Consolas,Liberation Mono,Menlo,monospace; - color: #528186; + color: #24292e; } code .string, code .number { @@ -835,13 +835,13 @@ code .keyword { font-weight: bold; } code .comment { - color: #adadcc; + color: #6a737d; } code .symbol { color: #000; } code .type { - color: #6e45a5; + color: #d73a49; } .lang-swift .number { From e687a0e7df12d155c0cfd71e204f327b40d607cb Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:19:13 -0500 Subject: [PATCH 126/150] fix body font color --- docs/style.css | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/style.css b/docs/style.css index a307b4b..56c6e84 100644 --- a/docs/style.css +++ b/docs/style.css @@ -78,7 +78,7 @@ td { } body { line-height: 1; - color: #000; + color: #24292e; background: #f6f6f6; } ol, From 89e986dc27f6e054e64915a80b8eaa426a0f40d3 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:52:52 -0500 Subject: [PATCH 127/150] finish getting started section --- README.md | 472 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 472 insertions(+) diff --git a/README.md b/README.md index d4fceaa..ae0e7bc 100644 --- a/README.md +++ b/README.md @@ -726,11 +726,477 @@ This example and many other important details about - [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). ### Connecting to Other Databases +Contexture depends on it's providers to be able to know how to +translate from the Contexture DSL to the specific DSL that each +database needs. Because of this, to connect to other databases you +will need to create a new Provider. + ### Simple Search Box +Building a simple search box consists of a single input field tied to +any of the fields that any record might have in the specified database +or index. To be able to make this text input tied to the search +structure, we will need to bring `contexture-client` in. With that in +mind, and the knowledge we already have of contexture, we can define +the following tasks: + +1. Create a New Contexture Search Function. +2. Create a Web Server With a Search Endpoint +3. Write a Search Tree. +4. Make the Search Tree Aware of Contexture. +5. Write a Text Input. + +Let's dive in. + +1. Creating a New Conteture Search Function + +Just as how we saw in the previous pages, creating a new Contexture +search function is about setting up the `contexture` package's default +export with `schemas` and `providers`. In this case, we'll use the +contexture-mongo approach in the following file (let's call it +`search.js`): + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient + +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} + +module.exports = {} + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + module.exports.search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +Note that we're now exporting the search function with +`module.exports.search = Contexture(...`. You can also note that we +specified the schema's name to be `collectionNameSchema`, and that any +search using this schema will be using MongoDb's `collectionName` +collection. + +2. Create a Web Server With a Search Endpoint + +If we want to separate the direct access to the database from the +client, the previous code should live in the server. Our next step is +to expose a `/search` endpoint so our future client can reach this +function. In this section, we'll write a simple web server with +`express` to satisfy our needs. + +2.1. Installing the Dependencies + +You'll need to install `express` and `body-parser` at the root of your +project, as follows: + + `npm install --save express body-parser` + +2.2. Writing the Web Server + +Once you have the dependencies installed, you can set up the server +with the following code: + +```javascript +let express = require('express') +let bodyParser = require('body-parser') +let search = require('./search') +let app = express() + +// create application/json parser +let jsonParser = bodyParser.json() + +app.post('/search', jsonParser, (req, res) => { + if (!req.body || !req.body.search) return res.sendStatus(400) + search(req.body.search).then((err, result) => { + if (err) return res.send(401, err) + res.send(200, result) + }) +}) + +app.listen(3000) +``` + +You can read more about these topics in the following links: + +- [Express Documentation](https://expressjs.com/en/api.html). +- [body-parser repository](https://github.com/expressjs/body-parser). + +3. Writing a Search Tree + +Having the DSL processor available through a web server endpoint, we +can follow up with the structure of the search interface itself. We'll +conceptualize this by writing the Contexture DSL itself. + +Let's use the same `searchTree` that we used in our first +script: + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} +``` + +Keep in mind that we'll be using `collectionNameSchema` since we +already defined a schema with that name on the server's `search.js` +file. + +4. Make the Search Tree Aware of Contexture + +Having a search tree, we will need `contexture-client` to make it +smart enough for the user interface. Let's make sure we have it +available in our project with: + + `npm install --save contexture-client` + +We will also use MobX to make it easier to notify our component that +the state has changed. For that purpose, we will need to install +`mobx` using NPM: + + `npm install --save mobx` + +Since we're heavy users of MobX, `contexture-client` already +provides an adapter that we can use out of the box. Knowing this, +let's prepare our `contexture-client` to work well with +`mobx`, as follows: + + +```javascript +let ContextureClient = require('contexture-client') +let ContextureMobx = require('contexture-react/dist/utils/contexture-mobx') + +let types = ContextureClient.exampleTypes +let service = async search => ({ + data: await postData('/sarch', { search }) +}) + +let Contexture = ContextureMobx({ + types, + service, +}) +``` + +Note that our service function will be the one responsible for sending +the `searchTree` we previously defined to the DSL processor, we can +wrap the search tree into a smart object that will later react to both +the user input, and the search results. + +```javascript +let contextureSearchTree = Contexture(searchTree) +``` + +You can read more about these topics here: + +- [MobX](https://mobx.js.org/). +- [MobX Observers](https://mobx.js.org/refguide/observer-component.html). + +5. Writing a Text Input + +Having the search tree ready allows us to write a `mobx` observer +component that will receive the tree and react to the result changes +immediately. Here's an example: + +``` +let { observer } = require('mobx') +let SearchQuery = observer(({ tree }) => + { + tree.getNode(['root', 'namequery']).value = e.target.value + }} + /> + +let SearchResults = observer(({ tree }) => ( +
+ {JSON.stringify(tree.getNode(['root', 'results']).context.response.results)} +
+)) +``` + +Which we would render this way: + +```javascript + + +``` + +This will generate a text input that will trigger a new search every +time the contents of the input change. This new search will get the +results and show the results in a JSON form (because of the +`JSON.stringify` part). Ideally, you will not render them in a JSON +form, but render them using a list component or table. + ### Your First Filter +In the previous page, we wrote a simple search user interface where a +single input would retrieve results. Now, we will add a simple filter +that will allow us to get results within a given range. + +#### Adding the Filter to the Tree + +The way we will add the filter is by adding a node to the tree. This +node is going to have a type of `number`, which asks for a `field`, a +`min` and a `max` values. + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'numberrange', + type: 'number', + field: 'age', + min: 0, + max: 100 + }, { + key: 'results', + type: 'results' + }] +} +``` + +With the `numberrange` node added, we can create another component +with two inputs that will allow us to filter results with the range +that the user specifies: + +```javascript +let RangeComponent = observer(({ tree }) => ( +
+ Minimum: + { + tree.getNode(['root', 'numberrange']).min = e.target.min + }} + /> +
+ Maximum: + { + tree.getNode(['root', 'numberrange']).max = e.target.max + }} + /> +
+)) +``` + +And that's it! Rendering this component will make our search aware of +any change the user might desire on the minimum and maximum ages they +might want to use to filter the available results. + ### Discovering the Database + +One of the great things about Contexture is that it can be used to +discover the database. In this page, we'll see how to write a filter +that not only allows the user to refine their search, but that also +shows information about our data that might not be obvious by looking +at the results directly. + +### The Facet Filter + +The `facet` type is just like any other type in the tree. It requires +a unique key and some other properties. In contrast to previously seen +types, such as `text` and `number`, facet doesn't require specific +values, but instead it asks for two properties: `field`, and +optionally `size`. This type is incredibly useful because besides +filtering the results based on the `value` (or `values`) the user +might have chosen, it retrieves from the database a list of available +values! The `facet` type can be very well represented as a list of +checkboxes, ready for users to pick for one or more values. Let's see +how we can use it. + +#### Adding the Facet Filter to the Tree + +To add the `facet` filter to the tree, we simply add it to the +structure we already had. For example: + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'numberrange', + type: 'number', + field: 'age', + min: 0, + max: 100 + }, { + key: 'cityfacet', + type: 'facet', + field: 'city', + value: 'Orlando', // Optional + // We could have instead `values` with ['Orlando', 'Miami'] + }, { + key: 'results', + type: 'results' + }] +} +``` + +Once we have it in the tree, we can simply create another component to +allow users to set a value to this type. In this case, however, we +will be retrieving the available values from a field that is +automatically inserted to this part of the tree. Let's make sure we +have the data before we render the components. + +#### Fetching the Available Options + +Having the facet node already added to the tree, it's only matter of +running a first search before we render the components to actually get +the possible results from the `facet` type. We can achieve this by +calling to `contexture-client`'s `refresh` function: + +```javascript +// This would go after: +// let contextureSearchTree = Contexture(searchTree) +contextureSearchTree.refresh(['root']) +``` + +Now, the available options will be ready to be used at: +`contextureSearchTree.getNode(['root', 'cityfacet']).context.options`. +So we can write our facet component this way: + +```javascript +let toggleValue = (array, value) => { + if (array.indexOf(value) > -1) { + let copy = array.slice() // Making sure it's not a MobX Array + copy.splice(array.indexOf(value), 1) // removing value from the array + array.replace(copy) // MobX Arrays have this method to replace the array's inner values + } else { + array.push(value) + } +} +let RangeComponent = observer(({ tree }) => ( +
+ Select One or More: + {tree.getNode(['root', 'cityfacet']).context.options.map(option => ( +
+ -1} + onChange={() => toggleValue(tree.getNode(['root', 'cityfacet']).values, option.value)} + /> + +
+ ))} +
+)) +``` + +Including this component in our search interface will show the current +available cities for the search results we have so far, and will allow +users to filter the search even more by letting them pick any (or many) +of the available cities. + ### IMDB Index +In our Contexture React repository, we've created a Storybook live +example that uses Contexture (with no server whatsoever, everything in +the client) to query and discover a public ElasticSearch index with +data similar to the data that IMDB might contain. + +- [Contexture React Repository](https://github.com/smartprocure/contexture-react). +- More information about [Storybook](https://github.com/storybooks/storybook). +- [The IMDB Live Index Explorer with Contexture](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). +- [The IMDB Live Index Explorer Source Code](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). + +Let's see how we can use this tool. + +#### Picking a Field + +The tool initially shows you a button named `Pick a Field`: + +![](https://i.imgur.com/CWz4Moy.png) + +This button will allow you to select any field on which you might want +to run a search. If you click it, you'll see the following fields: + +![](https://i.imgur.com/RR6vMP1.png) + +If we pick `Actors`, for example, the search will trigger and results +will appear: + +![](https://i.imgur.com/kep1LIo.png) + +Now, let's click on `Select Type` and click on `Facet`. A moment +afterwards, we will get a list of checkboxes with the most common +actors among the results: + +![](https://i.imgur.com/2Gq5FEh.png) + +We can see that the actor that has made the most movies (among the +ones idexed) is Naveen Andrews. We can also see the number of movies +this and other actors have made, and the total actors in the database +(of which we can only see 10 in this screenshot). Under this list of +actors, you'll see a text saying `View More`. If you click it, the +next 10 actors will appear (in order). + +At the bottom of the page, you'll see a button labeled `Add Filter`. +Clicking it will add another interactive component to the website, +which will say `Click to add AND`. + +![](https://i.imgur.com/sRHN04n.png) + +Clicking the `Click to add AND` button will show us again a `Pick a +Field` button: + +![](https://i.imgur.com/oA4LOEK.png) + +So we can start again. Let's say we pick the `Genre` field, and we +click for the `Facet` type again. A bit later, the list of ordered +genres will appear: + +![](https://i.imgur.com/nfbVlQ2.png) + +And that's it! We're discovering the database with a very simple and +unpolished interface. By this point you might be curious on what +components we're using to do all this magic. There's really no trick, +just follow us through the tutorial. You can also skip some steps if +you're particularly interested in: + +- [Contexture React Repository](https://github.com/smartprocure/contexture-react). + ## Querying ### Contexture DSL ### Interactive Queries @@ -794,7 +1260,13 @@ This example and many other important details about ### Contexture React ## Examples +- [contexture-site](https://contexture.site) - Showcase of Contexture searches adapting to diverse datasets such as: car crashes, restaurant locations, COVID-19, SAT scores, school grants, bank failures. +- [contexture-imdb](https://github.com/smartprocure/contexture-imdb) - An example usage of Contexture to present a search interface for an ElasticSearch index of movie records based on IMDB. +- [GovSpend](https://app.govspend.com/) - A Contexture web-based tool that makes it easy for government agencies to find a product's best price, identify and validate vendors, request quotes and connect with peers. + ## Contributing Guide +See the [SmartProcure Contributor Coding Guidelines](https://github.com/smartprocure/contributor-guide). ## License +Contexture is licensed under [the MIT License](https://github.com/smartprocure/contexture/blob/master/LICENSE). From 01b994b33416d3a7f3324b7f11c9741d141271df Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 20:57:06 -0500 Subject: [PATCH 128/150] fix image sizes --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index ae0e7bc..021e4c2 100644 --- a/README.md +++ b/README.md @@ -1147,23 +1147,23 @@ Let's see how we can use this tool. The tool initially shows you a button named `Pick a Field`: -![](https://i.imgur.com/CWz4Moy.png) + This button will allow you to select any field on which you might want to run a search. If you click it, you'll see the following fields: -![](https://i.imgur.com/RR6vMP1.png) + If we pick `Actors`, for example, the search will trigger and results will appear: -![](https://i.imgur.com/kep1LIo.png) + Now, let's click on `Select Type` and click on `Facet`. A moment afterwards, we will get a list of checkboxes with the most common actors among the results: -![](https://i.imgur.com/2Gq5FEh.png) + We can see that the actor that has made the most movies (among the ones idexed) is Naveen Andrews. We can also see the number of movies From e1cfef06ab63d191cd5322a9de64f098c5d1a90b Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:02:06 -0500 Subject: [PATCH 129/150] fix w,h of images --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 021e4c2..69509a8 100644 --- a/README.md +++ b/README.md @@ -1147,23 +1147,23 @@ Let's see how we can use this tool. The tool initially shows you a button named `Pick a Field`: - + This button will allow you to select any field on which you might want to run a search. If you click it, you'll see the following fields: - + If we pick `Actors`, for example, the search will trigger and results will appear: - + Now, let's click on `Select Type` and click on `Facet`. A moment afterwards, we will get a list of checkboxes with the most common actors among the results: - + We can see that the actor that has made the most movies (among the ones idexed) is Naveen Andrews. We can also see the number of movies From 50143019674b3fb353a271b3589d11bada0e2bda Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:04:49 -0500 Subject: [PATCH 130/150] fix all images --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 69509a8..7e2183d 100644 --- a/README.md +++ b/README.md @@ -1176,18 +1176,18 @@ At the bottom of the page, you'll see a button labeled `Add Filter`. Clicking it will add another interactive component to the website, which will say `Click to add AND`. -![](https://i.imgur.com/sRHN04n.png) + Clicking the `Click to add AND` button will show us again a `Pick a Field` button: -![](https://i.imgur.com/oA4LOEK.png) + So we can start again. Let's say we pick the `Genre` field, and we click for the `Facet` type again. A bit later, the list of ordered genres will appear: -![](https://i.imgur.com/nfbVlQ2.png) + And that's it! We're discovering the database with a very simple and unpolished interface. By this point you might be curious on what From 2847e1577e767a50d874c8c860b48927eb8efa65 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:12:46 -0500 Subject: [PATCH 131/150] fix unneeded ticks --- README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 7e2183d..d88b75c 100644 --- a/README.md +++ b/README.md @@ -214,31 +214,31 @@ Once you have NodeJS and NPM installed, you'll need either a new folder for your new project with Contexture, or to go to an existing project, then run: - `npm install --save contexture` + npm install --save contexture #### Install Contexture Client To install the `contexture-client` you can run the following command in your project's root folder: - `npm install --save contexture-client` + npm install --save contexture-client #### Install Contexture ElasticSearch To install the `contexture-elasticsearch` you can also run the following command in your project's root folder: - `npm install --save contexture-elasticsearch` + npm install --save contexture-elasticsearch #### Install Contexture Mongo To install the `contexture-mongo` you can also run the following command in your project's root folder: - `npm install --save contexture-mongo` + npm install --save contexture-mongo #### Install Contexture React To install the `contexture-react` you can also run the following command in your project's root folder: - `npm install --save contexture-react` + npm install --save contexture-react ### A First Contexture Script With everything installed, let's see a simple script that runs a @@ -803,7 +803,7 @@ function. In this section, we'll write a simple web server with You'll need to install `express` and `body-parser` at the root of your project, as follows: - `npm install --save express body-parser` + npm install --save express body-parser 2.2. Writing the Web Server @@ -872,13 +872,13 @@ Having a search tree, we will need `contexture-client` to make it smart enough for the user interface. Let's make sure we have it available in our project with: - `npm install --save contexture-client` + npm install --save contexture-client We will also use MobX to make it easier to notify our component that the state has changed. For that purpose, we will need to install `mobx` using NPM: - `npm install --save mobx` + npm install --save mobx Since we're heavy users of MobX, `contexture-client` already provides an adapter that we can use out of the box. Knowing this, From faf2d4571744bbd4be5a0dbfba7fbedd05cd6c9a Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:14:14 -0500 Subject: [PATCH 132/150] try TOC --- README.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/README.md b/README.md index d88b75c..c1585d8 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,89 @@ This is the complete documentation for [Contexture](https://github.com/smartprocure/contexture). +## Table of Contents + +- [About Contexture](about/README.md) + - [What is Contexture](about/what-is-contexture.md) + - [Glossary of Terms](about/glossary-of-terms.md) + - [Map of Repos](about/map-of-repos.md) + - [Brief History](about/brief-history.md) + - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) +- [Getting Started](getting-started/README.md) + - [Project Setup](getting-started/setup.md) + - [Install Node 9 and NPM](getting-started/setup.md#installing-node-9-and-npm) + - [Install Contexture](getting-started/setup.md#installing-contexture) + - [Install Contexture Client](getting-started/setup.md#installing-contexture-client) + - [Install Contexture ElasticSearch](getting-started/setup.md#installing-contexture-elasticsearch) + - [Install Contexture Mongo](getting-started/setup.md#installing-contexture-mongo) + - [Install Contexture React](getting-started/setup.md#installing-contexture-react) + - [A First Contexture Script](getting-started/first-script.md) + - [Connecting to Elasticsearch & MongoDB](getting-started/connecting.md) + - [Connecting to ElasticSearch](getting-started/connecting.md#connecting-to-elasticsearch.md) + - [Connecting to MongoDB](getting-started/connecting.md#connecting-to-mongodb.md) + - [Connecting to Other Databases](getting-started/connecting-other-databases.md) + - [Simple Search Box](getting-started/simple-search-box.md) + - [Your First Filter](getting-started/your-first-filter.md) + - [Discovering the Database](getting-started/discovering-the-database.md) + - [IMDB Index](getting-started/imdb-example.md) +- [Querying](querying/README.md) + - [Contexture DSL](querying/contexture-dsl.md) + - [Interactive Queries](querying/interactive-queries/README.md) + - [Contexture Client](querying/interactive-queries/contexture-client.md) + - [Initializing the Contexture Client](querying/interactive-queries/contexture-client.md#initializing-the-contexture-client) + - [Context Tree](querying/interactive-queries/contexture-client.md#context-tree) + - [Tree and getNode](contexture-client.md#tree-and-getnode) + - [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) + - [Introduction to Reactors](querying/interactive-queries/reactors.md) +- [Types and Type Components](types/README.md) + - [DIY Types](types/diy-types.md) + - [How to Write a Provider Type](types/diy-types.md#how-to-wite-a-provider-type) + - [How to Write a Client Type](types/diy-types.md#how-to-wite-a-client-type) + - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) + - [The Example Types](types/diy-types.md#the-example-types) + - [ElasticSearch Example Types](types/elasticsearch-example-types.md) + - [Mongo Example Types](types/mongo-example-types.md) + - [Available React Components for Types](types/react-components.md) +- [Other Components](other-components/README.md) + - [Generally Useful Components](other-components/general.md) + - [Layout Components](other-components/layout.md) +- [Managing State](managing-state/README.md) + - [DIY State Management](managing-state/diy.md) + - [MobX](managing-state/mobx.md) + - [Redux (Coming Soon)](managing-state/redux.md) +- [Theming](theming/README.md) +- [Recommendations](recommendations/README.md) + - [Architecture](recommendations/architecture.md) + - [Client vs Server](recommendations/architecture.md#client-vs-server) + - [Type definitions](recommendations/architecture.md#type-definitions) + - [Scaling](recommendations/architecture.md#scaling) + - [Server Side Searches](recommendations/server-side-searches.md) + - [Searching On an Endpoint](recommendations/server-side-searches.md#searching-on-an-endpoint) + - [Caching](recommendations/server-side-searches.md#caching) + - [Client Side Searches](recommendations/client-side-searches.md) + - [Click to Search](recommendations/client-side-searches.md#click-to-search) + - [Real Time Searches](recommendations/client-side-searches.md#real-time-searches) + - [Cascading](recommendations/client-side-searches.md#cascading) +- [Under the Hood](under-the-hood/README.md) + - [Design Principles](under-the-hood/design-principles.md) + - [Contexture Core](under-the-hood/contexture-core.md) + - [Default Export](under-the-hood/contexture-core.md#default-export) + - [The Algorithm](under-the-hood/contexture-core.md#the-algorithm) + - [Utility Functions](under-the-hood/contexture-core.md#utility-functions) + - [Contexture Providers](under-the-hood/contexture-providers/README.md) + - [Contexture ElasticSearch](under-the-hood/contexture-providers/) + - [Contexture Mongo](under-the-hood/contexture-providers/) + - [Building Your Own Provider](under-the-hood/contexture-providers/building-your-own-provider.md) + - [Contexture Client](under-the-hood/contexture-client.md) + - [State Properties](under-the-hood/contexture-client.md#state-properties) + - [Lenses](under-the-hood/contexture-client.md#lenses) + - [Reactors in Detail](under-the-hood/contexture-client.md#reactors-in-detail) + - [Serialization](under-the-hood/contexture-client.md#serialization) + - [Traversals](under-the-hood/contexture-client.md#traversals) + - [Contexture React](under-the-hood/contexture-react.md) +- [Examples](examples/README.md) +- [Contributing Guide](#contributing-guide) +- [License](#license) + ## About Contexture ### What is Contexture People of the Internet, here we officialy introduce you to From 00220d11ee6c5d55a14b04be7f9b17b01983eb88 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:15:59 -0500 Subject: [PATCH 133/150] try toc 2 --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c1585d8..b216a8b 100644 --- a/README.md +++ b/README.md @@ -81,8 +81,8 @@ - [Traversals](under-the-hood/contexture-client.md#traversals) - [Contexture React](under-the-hood/contexture-react.md) - [Examples](examples/README.md) -- [Contributing Guide](#contributing-guide) -- [License](#license) +- [Contributing Guide](#-contributing-guide) +- [License](#-license) ## About Contexture ### What is Contexture From c302f27289933e4e97a7ce2e8db9c4ac4d644ebf Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:18:08 -0500 Subject: [PATCH 134/150] remove TOC --- README.md | 84 ------------------------------------------------------- 1 file changed, 84 deletions(-) diff --git a/README.md b/README.md index b216a8b..d88b75c 100644 --- a/README.md +++ b/README.md @@ -1,89 +1,5 @@ This is the complete documentation for [Contexture](https://github.com/smartprocure/contexture). -## Table of Contents - -- [About Contexture](about/README.md) - - [What is Contexture](about/what-is-contexture.md) - - [Glossary of Terms](about/glossary-of-terms.md) - - [Map of Repos](about/map-of-repos.md) - - [Brief History](about/brief-history.md) - - [Alternatives & Benchmarks](about/alternatives-benchmarks.md) -- [Getting Started](getting-started/README.md) - - [Project Setup](getting-started/setup.md) - - [Install Node 9 and NPM](getting-started/setup.md#installing-node-9-and-npm) - - [Install Contexture](getting-started/setup.md#installing-contexture) - - [Install Contexture Client](getting-started/setup.md#installing-contexture-client) - - [Install Contexture ElasticSearch](getting-started/setup.md#installing-contexture-elasticsearch) - - [Install Contexture Mongo](getting-started/setup.md#installing-contexture-mongo) - - [Install Contexture React](getting-started/setup.md#installing-contexture-react) - - [A First Contexture Script](getting-started/first-script.md) - - [Connecting to Elasticsearch & MongoDB](getting-started/connecting.md) - - [Connecting to ElasticSearch](getting-started/connecting.md#connecting-to-elasticsearch.md) - - [Connecting to MongoDB](getting-started/connecting.md#connecting-to-mongodb.md) - - [Connecting to Other Databases](getting-started/connecting-other-databases.md) - - [Simple Search Box](getting-started/simple-search-box.md) - - [Your First Filter](getting-started/your-first-filter.md) - - [Discovering the Database](getting-started/discovering-the-database.md) - - [IMDB Index](getting-started/imdb-example.md) -- [Querying](querying/README.md) - - [Contexture DSL](querying/contexture-dsl.md) - - [Interactive Queries](querying/interactive-queries/README.md) - - [Contexture Client](querying/interactive-queries/contexture-client.md) - - [Initializing the Contexture Client](querying/interactive-queries/contexture-client.md#initializing-the-contexture-client) - - [Context Tree](querying/interactive-queries/contexture-client.md#context-tree) - - [Tree and getNode](contexture-client.md#tree-and-getnode) - - [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) - - [Introduction to Reactors](querying/interactive-queries/reactors.md) -- [Types and Type Components](types/README.md) - - [DIY Types](types/diy-types.md) - - [How to Write a Provider Type](types/diy-types.md#how-to-wite-a-provider-type) - - [How to Write a Client Type](types/diy-types.md#how-to-wite-a-client-type) - - [How to Write a UI Component for a Type](types/diy-types.md#how-to-write-a-ui-component-for-a-type) - - [The Example Types](types/diy-types.md#the-example-types) - - [ElasticSearch Example Types](types/elasticsearch-example-types.md) - - [Mongo Example Types](types/mongo-example-types.md) - - [Available React Components for Types](types/react-components.md) -- [Other Components](other-components/README.md) - - [Generally Useful Components](other-components/general.md) - - [Layout Components](other-components/layout.md) -- [Managing State](managing-state/README.md) - - [DIY State Management](managing-state/diy.md) - - [MobX](managing-state/mobx.md) - - [Redux (Coming Soon)](managing-state/redux.md) -- [Theming](theming/README.md) -- [Recommendations](recommendations/README.md) - - [Architecture](recommendations/architecture.md) - - [Client vs Server](recommendations/architecture.md#client-vs-server) - - [Type definitions](recommendations/architecture.md#type-definitions) - - [Scaling](recommendations/architecture.md#scaling) - - [Server Side Searches](recommendations/server-side-searches.md) - - [Searching On an Endpoint](recommendations/server-side-searches.md#searching-on-an-endpoint) - - [Caching](recommendations/server-side-searches.md#caching) - - [Client Side Searches](recommendations/client-side-searches.md) - - [Click to Search](recommendations/client-side-searches.md#click-to-search) - - [Real Time Searches](recommendations/client-side-searches.md#real-time-searches) - - [Cascading](recommendations/client-side-searches.md#cascading) -- [Under the Hood](under-the-hood/README.md) - - [Design Principles](under-the-hood/design-principles.md) - - [Contexture Core](under-the-hood/contexture-core.md) - - [Default Export](under-the-hood/contexture-core.md#default-export) - - [The Algorithm](under-the-hood/contexture-core.md#the-algorithm) - - [Utility Functions](under-the-hood/contexture-core.md#utility-functions) - - [Contexture Providers](under-the-hood/contexture-providers/README.md) - - [Contexture ElasticSearch](under-the-hood/contexture-providers/) - - [Contexture Mongo](under-the-hood/contexture-providers/) - - [Building Your Own Provider](under-the-hood/contexture-providers/building-your-own-provider.md) - - [Contexture Client](under-the-hood/contexture-client.md) - - [State Properties](under-the-hood/contexture-client.md#state-properties) - - [Lenses](under-the-hood/contexture-client.md#lenses) - - [Reactors in Detail](under-the-hood/contexture-client.md#reactors-in-detail) - - [Serialization](under-the-hood/contexture-client.md#serialization) - - [Traversals](under-the-hood/contexture-client.md#traversals) - - [Contexture React](under-the-hood/contexture-react.md) -- [Examples](examples/README.md) -- [Contributing Guide](#-contributing-guide) -- [License](#-license) - ## About Contexture ### What is Contexture People of the Internet, here we officialy introduce you to From cc58b11eb9da95bf662b176c17d1cbfb5c7d4de1 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:24:56 -0500 Subject: [PATCH 135/150] finish querying section --- README.md | 239 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 239 insertions(+) diff --git a/README.md b/README.md index d88b75c..ea389b3 100644 --- a/README.md +++ b/README.md @@ -1199,13 +1199,252 @@ you're particularly interested in: ## Querying ### Contexture DSL +The Contexture DSL is a JavaScript object structure, or JSON +structure, that is composed of nested nodes that are equal at all +levels. Each node has a `key`, a `type` and many other optional +properties. The first node is called the root and is a node of type +`group`. Any node of type `group` can have many children. Each +children can be a node of any type. If any children is another group, +this children will probably have one or more other children nodes of +any type, and so on. + +Let's begin talking about the root. + +#### The Root + +The root of a Contexture Tree is a node of type `group`. Group types +are required to have a `key`, the `type: "group"` property, and an +array of children `children: [/* nodes */]`. Root groups also need an +extra property: then ame of the schema. In summary, this is how a root +node should look: + +```javascript +let searchTree = { + key: 'myRootNode', + type: 'group', + schema: 'mySchemaName', + children: [ + // Other nodes + ] +} +``` + +#### The Children +Each children will be an individual node. Each node will have at least +a unique `key` (unique per tree, but they can appear again in other +trees), and a `type`. Some types might require more properties. + +That's really all that it is for our DSL. The magic happens in the +types and the providers. As long as your types are defined properly, +and your providers build the correct queries and write them back in +the tree, Contexture will work and you'll be able to use the +Contexture DSL to build search interfaces of any complexity. + +**Note:** Many of the types have some common properties, like `field` +or `values`. The only reason these properties are common is because +they've made sense to be common for each one of our specific types, +not because they have something to share between types. Each type has +it's own properties and rules, so you should treat them as independent +structures. + ### Interactive Queries #### Contexture Client +Having a Contexture DSL by itself won't work since it's only a +language and not an interactive or working program in any sense. We +need something to automatically send this query to our DSL processor +(the main `contexture` repository). For this purpose, we provide the +`contexture-client`. + +The Contexture Client is responsible for keeping track of the changes +that happen on the Contexture DSL, either by the user or by the +Contexture core. The nodes in the Contexture DSL host values used for +filtering the results, but also hold other properties, such as the +search results, and a `_meta` object that you can use for debugging +purposes. In this page, we will sumarize how the Contexture Client +works with the Contexture DSL to provide real-time interactive +searches. + ##### Initializing the Contexture Client +The Contexture Client has a default export that needs to be called +first with some configuration properties before being able to process +any search. This is how you would normally initialize it: + +```javascript +let ContextureClient = require('contexture-client') +let Contexture = ContextureClient({ + service, + types, +}) +``` + +When Contexture Client determines the search query is ready to be +executed, it will call the `service` function provided with the whole +Contexture DSL. It expects to receive another full DSL that it parses +to then change the nodes again. As previously mentioned, an example +`service` would be: + +```javascript +let service = async search => ({ + data: await postData('/sarch', { search }) +}) +``` + +The `types` property is an object where each key is a type's name. +These type definitions are different than the ones from the servers +and allow our Contexture Client to do three things: + +- To know how to validate a node. +- To know what happens to a specific node (and it's suroundings) when + it changes. +- To know how to complemente any missing value on a specific node + (default values, so you can omit properties when you write each of + the nodes). + +Each one of the types will have the following properties: + +| Property Name | Type | Description | +| --- | --- | --- | +| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | +| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | +| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | + ##### Context Tree +Once you have Contexture Client initialized, we will be able to pass +the Contexture DSL into it. This will generate something we call the +`Context Tree`, which is an object with the following properties: + +| Name | Signature | Description | +| ---- | --------- | ----------- | +| add | `async (path, newNode) -> await searchCompleted` | Adds a node to the tree as a child of the specified path. You can await this for when updates settle and relevant searches are completed. | +| remove | `async path -> await searchCompleted` | Removes a node at the specified path. You can await this for when updates settle and relevant searches are completed. | +| mutate | `async (path, deltas) -> await searchCompleted` | Mutates the node at the given path with the new values. You can await this for when updates settle and relevant searches are completed. | +| getNode | `[path] -> node` | Lookup a node by a path (array of keys). | +| tree | tree | A reference to the internal tree. If you mutate this, you should dispatch an appropriate event. | + +You can read more about the instantiated client here: [contexture-client Run Time](https://github.com/smartprocure/contexture-client#run-time). + ##### Tree and getNode +Context Trees have two properties for nagivating through the +Contexture DSL: `tree` and `getNode`. The `tree` property will have +the DSL as it was received, plus the default properties set by each +one of the client types, and some state properties. On the other hand, +`getNode` is a quick way to access the tree by sending only the keys +of each node. For example, let's say we have the following DSL: + +```javascript +let searchTree = ContextureClient({ + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }] +}) +``` + +We are able to access the `namequery` node by simply calling: + +```javascript +let nameQueryNode = searchTree.getNode(['root', 'namequery']) +``` + ##### Mutate, Add and Remove +Once you have a Context Tree ready, you will trigger searches +automatically by invoking one of the following functions: `add`, +`remove`, `mutate`. More specifically: + +- If you want to run a search after changing some of the properties of + a specific node, you would call `mutate`. +- If you want to add a children to any given `group`, even the root + group, you would call `add`. +- If you want to remove a children to any given `group`, even the root + group, you would call `remove`. + +Let's see some examples: + +1. Mutate + +Mutate allows us to change properties on nodes and trigger search +afterwards. Here is an example: + +```javascript +await searchTree.mutate(['root', 'namequery'], { + value: 'new value' +}) +``` + +This will change the tree and produce a new set of results. If you're +wondering how to keep track of these changes, the simplest way to do +it is by using our MobX adapter, as shown in our [simple search +box](../../getting-started/simple-search-box.md) tutorial, or in +greater detail in our [Managing State](managing-state/README.md) docs. + +2. Add + +Having the previous search tree, we can add a children by doing: + +```javascript +await searchTree.add(['root'], { + key: 'results', + type: 'results' +}) +``` + +3. Remove + +We can remove the `results` node we just added by doing: + +```javascript +await searchTree.add(['root', 'results']) +``` + +Calling `mutate`, `add` or `remove` will trigger events not only for +the node that these functions are targetting, but also for nearby +nodes, depending on the types. + +Up next, we'll dig a bit into what are the client side reactors (the +rules that Contexture Client follows to know what other nodes are +relevant for each update). + #### Introduction to Reactors +When any node changes, depending on the type, it might be reasonable +to re-trigger a search call for other nodes. We call this process the +selection of `reactors`, where the possible reactors are only three: +`self`, `others` and `all`. + +Reactors should be specified in the Client Types, where each type will +have a specific reactor for each one of it's properties. For example +(taken from our client side [example +types](https://github.com/smartprocure/contexture-client/blob/master/src/exampleTypes.js)): + +```javascript +facet: { + reactors: { + values: 'others', + mode: 'others', + size: 'self', + optionsFilter: 'self', + sort: 'self', + } +} +``` + +The client side type defined above will be effective for any node with +type `facet`, where the properties `values` and `mode` will affect +only all the other nodes (and not itself), and the properties `size`, +`optionsFilter` and `sort` will affect only the specific `facet` node +and no other node. + +The one remaining reactor that isn't covered by that example is the +`all` reactor. The difference between `others` and `all` is that +`others` excludes the node where the change is happening, and `all` +includes all other nodes and the node where the change is happening +(effectively combining `self` and `others`). + ## Types and Type Components ### DIY Types From 46039f43d71f7d88fef611e817220ac5d67388ea Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:38:49 -0500 Subject: [PATCH 136/150] add types and type components section --- README.md | 1327 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1327 insertions(+) diff --git a/README.md b/README.md index ea389b3..fc292a4 100644 --- a/README.md +++ b/README.md @@ -1448,13 +1448,1340 @@ includes all other nodes and the node where the change is happening ## Types and Type Components ### DIY Types +The Contexture ecosystem provides a defined list of types that can be +used to perform a wide variety of different searches. Each type we +offer also has a respective component in our `contexture-react` repo. +We've made these components so you can quickstart your search interfaces! + +Even if our types are focused on the different search interfaces we +provide, our API is designed to allow you to build any type you might +need for any other possible use case you might encounter. + +We believe that making a generic framework will allow users to be +creative on their search solutions. Because of that, we will start this +document by explaining how to build your own types. + #### How to Write a Provider Type +Initializing Contexture Core +requires you to send a bunch of types per provider. A type on any +provider is just a valid string property name on the object that is +sent, accompanied with a corresponding value of a plain JavaScript +Object with one or more of the following properties: + +| Property | Type | Params (Type) | Return Value Type | What it does | +| --- | --- | --- | --- | --- | +| `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | +| `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | +| `result` | Function | Node (Object), Search (Function) | Promise | Allows running a direct search for this type before Comtexture sends the full seaech with the whole tree. | + +Once you have written a type, you can use it by sending it to an +existing Contexture Provider. It +should look more or less like: + +```javascript +let myType = { + hasValue: node => node.requiredProperty, + filter: node => ({ + providerQueryObject: { + value: node.requiredProperty + } + }) +} + +let provider = MyProvider({ + types: { + myType + } +}) +``` + +**Type names are not exclusive across providers**. You can define one +type called `myAwesomeType` in more than one provider and you'll be +able to keep the same required node properties, thus the same +`hasValue`. This allows us to provide the same API for several types, +and re-use code even if we switch the target database of the search. + +Once you have a type defined for one or more providers, you should +write the same type for `contexture-client`. + #### How to Write a Client Type +Contexture Client already provides a some types based on our +`Example Types` (more on that later.) These type definitions help +the client understand how a specific node affects every other node or +itself. + +To create a custom type, you will need to think on the behaviors you +might need for each one of the following properties: + +| Property Name | Type | Description | +| --- | --- | --- | +| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | +| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | +| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | + +More details about `contexture-client` types, their properties and +their reserved words can be seen on the README of +[contexture-client](https://github.com/smartprocure/contexture-client#client-types). + +The example types are already included in any instantiation +of Contexture Client's Contexture Tree. However, you can add any type +you need simply by extending the exposed `exampleTypes` with your own. +In the following snippet, we initialize a `ContextureTree` with the +available `exampleTypes`, and our new `myType`: + +```javascript +import * as ContextureClient from 'contexture-client' + +let tree = ContextureClient.ContextTree({ + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } +}, { + // Here we will have the underlying tree +}) +``` + #### How to Write a UI Component for a Type +Writing a user interface for any type can be as simple as writing an +HTML or JSX Element that will render or modify any property of the +any node of an existing Contexture Tree, for example, using our custom +type `myType`, we could write an input field that, onChange, will +write the field's value into the `requiredProperty`. For example: + +```javascript +// This is ES6+ and JSX + +import * as ContextureClient from 'contexture-client' + +let tree = ContextureClient.ContextTree( + { + service: myService, + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } + }, { + key: 'root', + join: 'and', + children: [{ + key: 'myNode', + type: 'myType', + }] + } +) + +let node = tree.getNode(['root', 'myNode']) + +let Component = ({ node }) => ( + { + node.requiredProperty = e.target.value + }} + /> +) +``` + +Now that you have a component you can render it and play with it, but +the component won't render by itself. + #### The Example Types +With the intention of providing practical examples of how to write +types, we decided to share some of the types we use in our production +applications. These types belong to two different database processors: +`contexture-elasticsearch` and `contexture-mongo`. + +**Example Types aren't the rule**. These types are only provided to +serve as a guide to build any other type you might need for your +application. You can also **extend our example types**, simply by +assigning new types as properties on the objects exposed by +`contexture-elasticsearch` and `contexture-mongo`. + ### ElasticSearch Example Types +Contexture is designed to target any database you might need. However, +so far we have only inplemented database providers for the only +databases that we use: ElasticSearch and Mongo. + +Most of our types have relevant components already written to +facilitate building search interfaces. As we progress over each one of +these types, we will show small examples of simple components written +to search with these types. We hope to provide enough information to +allow you to take as little as you need, or as much as you want, and +fulfill you expectations. + +Our ElasticSearch types are the following ones: + +#### Results Type + +The `results` node is a node that if present, idicates to Contextre +that the queries and aggregations should finally run and retrieve all +the filtered values. It allows some customization properties, as +shown below: + +| Property Name | Type | Description | +| --- | --- | --- | +| `page` | Number | Current page of results. | +| `pageSize` | Number | Total results per page. | +| `sortField` | String | Field used to sort the results. | +| `sortDir` | String | Direction of sorting (`asc`, `desc`) | + +#### Bool Type + +The bool type is intended to work as an ElasticSearch terms +aggregation with only one value for a single property. This is useful +for user interfaces with a checkbox to include or exclude a specific +field from a search (or a specific field-value pair). + +Here is the list of all properties this type uses: + +| Property Name | Type | Description | +| --- | --- | --- | +| field | String | Name of the field that we will be using to filter the search search. | +| value | String | Value of the field that will be used to filter the search. | + +Example input: + +```javascript +{ + type: 'bool', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + term: { + fieldName: true + } +} +``` + +You can read more about it in: + +- [Source code of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js). +- [Unit tests of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/bool.js). +- Elastic Search [Term Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html). + +#### Cardinality Type + +The Cardinality type serves to calculate an approximate count of the +distinct available values. This type only uses one property, `field`. +It returns it's values in the `context` of the node, rather than in +the `results` node. + +Example input: +```javascript +{ + type: 'cardinality', + field: 'Organization.Name.untouched' +} +``` + +Example output: +```javascript +{ + aggs: { + cardinality: { + cardinality: { + field: 'Organization.Name.untouched' + } + } + } +} +``` + +You can read more about it in: + +- [Source code of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/cardinality.js). +- [Unit tests of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/cardinality.js). +- Elastic Search [Cardinality Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html). + +#### Date Type + +The Date type is used to specify a range of dates that will be used to +filter the available results. This range of dates can be specified by +a string formatted date (`YYYY-MM-DD`) or by a small set of possible +humanly readable date ranges. Details follow: + +| Property Name | Type | Description | +| --- | --- | --- | +| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | +| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | +| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | +| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | + +Example input 1: +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25' +} +``` + +Example output 1: +```javascript +{ + range: { + fieldName: { + gte: '2016-04-25', + format: 'dateOptionalTime' + } + } +} +``` + +Example input 2: +```javascript +{ + type: 'date', + field: 'fieldName', + from: 'thisQuarter', + useDateMath: true, +} +``` + +Example output 2: +```javascript +{ + range: { + fieldName: { + gte: moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD'), + lte: moment.utc(datemath.parse(`${moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD')}||+3M-1d/d`)).format('YYYY-MM-DD'), + format: 'dateOptionalTime' + } + } +} +``` + +You can read more about it in: + +- [Source code of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). +- [Unit tests of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +- Elastic Search [Range QUery](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html). + +#### Date Histogram Type + +The `dateHistogram` type is very useful for retrieving the number of +results within specific periods of time. The idea here is to end up +building a chart showing how records have changed over time, for +example by it's creation date or some other date property. This type +is able to do this by running a nested stats aggregation inside a +dateHistogram aggregation, while also supporting for tweaking min/max +bounds. + +This type returns it's values in the `context` of the node, rather +than in the `results` node. + +| Property Name | Type | Description | +| --- | --- | --- | +| `key_field` | String | What might be considered a good candidate for the X axis of the chart. | +| `value_field` | String | What might be considered a good candidate for the Y axis of the chart. | +| `interval` | | String | Available expressions for interval: `year` (`1y`), `quarter` (`1q`), `month` (`1M`), `week` (`1w`), `day` (`1d`), `hour` (`1h`), `minute` (`1m`), `second` (`1s`). | +| `boundsRange_min` | Date or String | Lower date limit that will be considered to filter the possible results. | +| `boundsRange_max` | Date or String | Upper date limit that will be considered to filter the possible results. | +| `boundsRange_useDateMath` | Boolean | If set to `true`, the min and max ranges will be valid as strings representative of dates, and will be formatted by NPM's library [`@elastic/datemath`](https://github.com/elastic/datemath-js). | + +Example input: +```javascript +{ + type: 'dateHistogram', + key_field: 'PO.IssuedDate', + value_field: 'LineItem.TotalPrice' +} +``` + +Example output: +```javascript +{ + aggs: { + max_date: { + max: { + field: 'PO.IssuedDate', + }, + }, + min_date: { + min: { + field: 'PO.IssuedDate', + }, + }, + twoLevelAgg: { + date_histogram: { + field: 'PO.IssuedDate', + interval: 'year', + min_doc_count: 0, + }, + aggs: { + twoLevelAgg: { + stats: { + field: 'LineItem.TotalPrice', + }, + }, + }, + }, + }, +} +``` + +You can read more about it in: + +- [Source code of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). +- [Unit tests of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). +- [Two Level Aggregations in ElasticSearch](https://www.elastic.co/blog/intro-to-aggregations-pt-2-sub-aggregations). +- [Date Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html). + +#### Exists Type + +The `exists` type is used to check wether some property exsits or not. +It requires only two fields: `field` and `value`. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The target field we want to check. | +| `value` | Boolean | the value we want to check. Normally true or false. | + +Example input: +```javascript +{ + type: 'exists', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + exists: { + field: 'fieldName' + } +} +``` + +You can read more about it in: + +- [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). +- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). +- Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). + +#### Facet Type + +The `facet` type represents a list of dynamic choices, e.g. a checkbox +list filter. We achieve this by running an ElasticSearch terms +aggregation. We provide a way to limit the number of results that you +will receive with the `size` property, so that large queries can +safely be used. For that same purpose, the property `optionsFilter` is +given, so that search queries can filter the results with a string. + +Facet returns it's values in the `context` of the node, rather than in +the `results` node. + +| Property | Type | Default | Description | +| ---- | ---- | ------- | ----------- | +| `field` | String | None, *required* | The field it's operating on. | +| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | +| `values` | Array (of strings) | `[]` | Already selected values. | +| `fieldMode` | String (`autocomplete`, `word` or `suggest`) | `autocomplete` | Whether to look at the entire field (`autocomplete`), the analyzed words in the field (`word`), or magic suggestions (`suggest`). | +| `size` | Number | 12 | How many options to return. | +| `cardinality` | Number | 5000 | Precision threshold override. | +| `includeZeroes` | Boolean | false | If true, it will include options with 0 matching documents (aka `min_doc_count: 0`) | +| `optionsFilter` | String | '' | Filters the options further, e.g. a find box above a checkbox list | +| `caseSensitive` | Boolean | false | Whether options filter is case sensitive. | +| `sort` | String (`term` or `count`) | `count` | Sort results alphabetically or by count of matching records. | + +Example input 1: +```javascript +{ + type: 'facet', + field: 'fieldName', + values: ['abc', '123'] +} +``` + +Example output 1: +```javascript +{ + terms: { + 'fieldName.untouched': ['abc', '123'] + } +} +``` + +Example input with exclude: +```javascript +{ + type: 'facet', + field: 'fieldName', + mode: 'exclude', + values: ['abc', '123'], +} +``` + +Example output with exclude: +```javascript +{ + bool: { + must_not: { + terms: { + 'fieldName.untouched': ['abc', '123'], + }, + }, + }, +} +``` + +You can read more about it in: + +- [Source code of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +- [Unit tests of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). + +#### Geo Type + +The `geo` type represents a geographic radius search. It needs a +geocodeLocation service passed in to it. We currently assume that you +will be using a google maps geocoder search. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String (required) | The field it's operating on | +| `location` | String (required) | Location to geocode (e.g. an address, businessname, anything the google geocode can take) | +| `radius` | Number (required) | Radius in miles | +| `operator` | String (either `within` or `not within`) | Whether the filter forces inclusion or exclusion (defaults with `within`). | + +Example input: +```javascript +{ + type: 'geo', + field: 'fieldName', + location: 'SmartProcure', + radius: 10, + operator: 'within', +} +``` + +Example output: + +```javascript +{ + geo_distance: { + fieldName: '26.3170479,-80.1131784', + distance: '10mi', + }, +} +``` + +You can read more about it in: + +- [Source code of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/geo.js). +- [Unit tests of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/geo.js). +- [ElasticSearch's Geo Distance Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-query.html). + +#### Number Type + +Number represents a number range with inclusive bounds. This type +provides the ability to determine the best range values based on +percentile interval and range threshold. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | + +Some Notes: +1. An empty value as the upper boundary represents infinity. +2. An empty value as the lower boundary represents negative infinity. +3. Zero has to be respected as a boundary value. +4. If findBestRange is true it will return the best min and max range. + +Example input: +```javascript +{ + type: 'number', + field: 'fieldName', + min: 500, + max: 1000, +} +``` + +Example output: + +```javascript +{ + range: { + fieldName: { + gte: 500, + lte: 1000, + }, + }, +} +``` + +You can read more about it in: + +- [Source code of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). +- [Unit tests of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). + +#### Number Range Histogram Type + +The type `numberRangeHistogram` represents a number range with inclusive bounds. This type +returns feedback in the form of histogram and statistical data. +This type returns it's values in the `context` of the node, rather +than in the `results` node. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | +| `percentileInterval` | Number | Used to group the results based on how many of the records are within each one of the sections given by the interval, from the `min` value, up to the `max` value. | + +Some Notes: +1. An empty value as the upper boundary represents infinity. +2. An empty value as the lower boundary represents negative infinity. +3. Zero has to be respected as a boundary value. + +- [Source code of the type: numberRangeHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/numberRangeHistohgram.js). +- [Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/search-aggregations-bucket-histogram-aggregation.html). + +#### Query Type + +Query represents a raw elasticsearch query_string. It's mostly used to +provide a simple sarch box on the user interface. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | +| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches (accepting values if they contain the given query, even if the casing doesn't match). | + +Example input: +```js +{ + type: 'query', + field: 'fieldName', + query: 'cable', + exact: true, +} +``` + +Example output: +```js +{ + query_string: { + query: 'cable', + default_operator: 'AND', + default_field: 'fieldName.exact', + analyzer: 'exact', + }, +} +``` + +You can read more about it in: + +- [Source code of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +- [Unit tests of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). + +#### Text Type + +Text implements raw text analysis like starts with, ends with, etc. +These are generally regex queries. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `join` | String | Either `any`, `all` or `none`. | +| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | +| `values` | Array | Array containing all the words that want to be used as inputs. | + +Example input: +```js +{ + type: 'text', + field: 'fieldName', + join: 'any', + operator: 'contains', + values: ['laserjet', 'printer'], +} +``` + +Example output: +```js +{ + query_string: { + default_field: 'fieldName', + default_operator: 'OR', + query: '"laserjet" "printer"', + }, +} +``` + +You can read more about it in: + +- [Source code of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/text.js). +- [Unit tests of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/text.js). + +#### Other ElasticSearch Example Types + +For more informaion about other available example types, please check: + + ### Mongo Example Types +Most of our types have relevant components already written to +facilitate building search interfaces. As we progress over each one of +these types, we will show small examples of simple components written +to search with these types. We hope to provide enough information to +allow you to take as little as you need, or as much as you want, and +fulfill you expectations. + +Our MongoDb types are the following ones: + +#### Results Type + +The `results` node is a node that if present, idicates to Contextre +that the queries and aggregations should finally run and retrieve all +the filtered values. It allows some customization properties, as +shown below: + +| Property Name | Type | Description | +| --- | --- | --- | +| `page` | Number | Current page of results. | +| `pageSize` | Number | Total results per page. | +| `sortField` | String | Field used to sort the results. | +| `sortDir` | String | Direction of sorting (`asc`, `desc`) | + +#### Date Type + +The Date type is used to specify a range of dates that will be used to +filter the available results. This range of dates can be specified by +a string formatted date (`YYYY-MM-DD`) or by a small set of possible +humanly readable date ranges. Details follow: + +| Property Name | Type | Description | +| --- | --- | --- | +| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | +| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | +| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | +| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | + +Example input: +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25' +} +``` + +Example output: +```javascript +{ + fieldName: { + $gte: new Date('2016-04-25'), + }, +} +``` + +You can read more about it in: + +- [Source code of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). +- [Unit tests of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). +- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). +- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). + +#### Exists Type + +The `exists` type is used to check wether some property exsits or not. +It requires only two fields: `field` and `value`. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The target field we want to check. | +| `value` | Boolean | the value we want to check. Normally true or false. | + +Example input: +```javascript +{ + type: 'exists', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + $and: [ + { + fieldName: { + $exists: true, + $ne: '', + }, + }, + { + fieldName: { + $ne: null, + }, + }, + ], +} +``` + +You can read more about it in: + +- [Source code of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/exists.js). +- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/exists.js). +- MongoDb's [$exists](https://docs.mongodb.com/manual/reference/operator/query/exists/). + +#### Facet Type + +The `facet` type represents a list of dynamic choices, e.g. a checkbox +list filter. We achieve this by running an ElasticSearch terms +aggregation. We provide a way to limit the number of results that you +will receive with the `size` property, so that large queries can +safely be used. For that same purpose, the property `optionsFilter` is +given, so that search queries can filter the results with a string. + +Facet returns it's values in the `context` of the node, rather than in +the `results` node. + +| Property | Type | Default | Description | +| ---- | ---- | ------- | ----------- | +| `field` | String | None, *required* | The field it's operating on. | +| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | +| `values` | Array (of strings) | `[]` | Already selected values. | +| `size` | Number | 10 | How many options to return. | + +Example input: +```javascript +{ + type: 'facet', + field: 'fieldName', + values: ['abc', '123'] +} +``` + +Example input with exclude: +```javascript +{ + type: 'facet', + field: 'fieldName', + mode: 'exclude', + values: ['abc', '123'], +} +``` + +You can read more about it in: + +- [Source code of the type: facet](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). + +#### Number Type + +Number represents a number range with inclusive bounds. This type +provides the ability to determine the best range values based on +percentile interval and range threshold. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | + +Some Notes: +1. An empty value as the upper boundary represents infinity. +2. An empty value as the lower boundary represents negative infinity. +3. Zero has to be respected as a boundary value. +4. If findBestRange is true it will return the best min and max range. + +Example input: +```javascript +{ + type: 'number', + field: 'fieldName', + min: 500, + max: 1000, +} +``` + +Example output: + +```javascript +{ + fieldName: { + $gte: 500, + $lte: 1000 + } +} +``` + +You can read more about it in: + +- [Source code of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). +- [Unit tests of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). +- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). +- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). + +#### Text Type + +Text implements raw text analysis like starts with, ends with, etc. +These are generally regex queries. + +| Property Name | Type | Description | +| --- | --- | --- | +| `field` | String | The field we will be using to filter the results. | +| `join` | String | Either `any`, `all` or `none`. | +| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | +| `values` | Array | Array containing all the words that want to be used as inputs. | + +Example input: +```js +{ + type: 'text', + field: 'fieldName', + join: 'any', + operator: 'contains', + values: ['laserjet', 'printer'], +} +``` + +Example output: +```js +{ + $or: [{ + fieldName: { + $regex: 'laserjet', + $options: 'i' + } + }, { + fieldName: { + $regex: 'printer', + $options: 'i' + } + }] +} +``` + +You can read more about it in: + +- [Source code of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/text.js). +- [Unit tests of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/text.js). + +#### Other MongoDb Example Types + +For more informaion about other available example types, please check: + + ### Available React Components for Types +A huge part of working with advanced search interfaces is identifying +how to portray a specific search filter that you might want to use. +We've talked about very simple components that react to changes on the +tree state, but now we'll see how to make it easier for more complex +nodes to gather the inputs necessary, and also to show the results +correctly. Here we'll see components specifically crafted for some of +our providers' example types. + +**Notes:** +- Keep in mind that the theme on these components is purely +optional. +- Please be aware that when we refer to `Component`, we mean a + Function that returns a valid JSX Element. You can read more here: + [Components and + Props, on the ReactJS docs](https://reactjs.org/docs/components-and-props.html). + +#### Query + + + +The Query component is probably the most commonly needed for search +interfaces. It's an input field that allows you to filter results if +the text matches part of the value of a specfic property on any of the +records. + +Here's how you write a node of type `query` in your _searchTree_: +```javascript +{ + key: 'searchQuery', + type: 'query', + field: 'title', + query: '' +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `query` | String | No | Search query that should be visible in the component. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. + +Here's how you write your component: +```javascript +let Query = require('contexture-react/dist/exampleTypes').Query +// ... +// Later, on your render function, or where you put your components: + +``` + +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `TextInput` | Component | `input` | Text input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +- [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). +- [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). +- [(MongoDb Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/query.js). +- [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). + +#### Date + + + +_(Same goes with the right, not adding another screenshot to avoid +consuming more space.)_ + +The Date component helps by allowing users to filter the data to +obtain results within a specific range of dates. It consists of only +two date pickers. Here's how you write a node of type `date` in your +_searchTree_: +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25', + to: '2017-05-26' +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `from` | Date or String (`YYYY-MM-DD` format) | Yes | The initial date of our timeframe. | +| `to` | Date or String (`YYYY-MM-DD` format) | No | The final date of our timeframe. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. + +Here's how you write your component: +```javascript +let DateComponent = require('contexture-react/dist/exampleTypes').Date +// ... +// Later, on your render function, or where you put your components: + +``` + +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `DateInput` | Component | `x => ` | The component that wraps each one of the inputs where the dates end up written by the date-picker. | + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). +- [(ElasticSearch Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +- [(MongoDb Provider) Source code of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). +- [(MongoDb Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). +- [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). + +#### DateHistogram + + + +The DateHistogram component is about representing how many records +were found during what periods of time. This component currently +doesn't offer interactive features, but you could use it as +inspiration to write another one that would allow you to dive in these +date ranges to see what records appear for each one of them. + +Here's how you write a node of type `dateHistogram` in your _searchTree_: +```javascript +{ + key: 'releases', + type: 'dateHistogram', + key_field: 'released', + value_field: 'imdbVotes', + interval: '3650d', +} +``` + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. + +Since this component doesn't need any property from the node itself, +but only from the results, here's how you write your component: +```javascript +let DateHistogram = require('contexture-react/dist/exampleTypes').DateHistogram +let formatYear = x => new Date(x).getFullYear() + 1 +// ... +// Later, on your render function, or where you put your components: + +``` + +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `background` | Function | `(record, max) => '#ccc'` | A function that returns the background color that is used to render each one of the bars in the resulting chart. | +| `height` | Number | `100` | Specifies the max height of the whole chart. | +| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | +| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | +| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). +- [(ElasticSearch Provider) Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). +- [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). + +#### Facet + + + +The Facet component allows users to filter the results by picking a +specific common option among all the values that the records might +have for a specific field. + +Here's how you write a node of type `facet` in your _searchTree_: +```javascript +{ + key: 'facet', + type: 'facet', + values: ['a'], +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `values` | Array | Yes | Array of selected values. To have a value selected by default, you will need to know in advance which value to put here, otherwise you can leave this with an empty array and let users select the values themselves. | +| `size` | Number | No | Max number of options to display. The default is 10. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. + +Here's how you write your component: +```javascript +let Facet = require('contexture-react/dist/exampleTypes').Facet +// ... +// Later, on your render function, or where you put your components: + +``` + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `hide.facetFilter` | Object (with an optional single property, `facetFilter`) | `{}` | Allows you to hide the text input that helps on searching for the available options. This text input is very valuable when the results are larger than the available visible results. | +| `TextInput` | Component | `input` | Allows you to customize the text input. | + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +- [(ElasticSearch Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +- [(MongoDb Provider) Source code of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). +- [(MongoDb Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/facet.js). +- [Source code of the Facet component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Facet.js). + +#### Number + + + +The Number component allows users to specify a numeric range to filter +the data based on the available results which values fit within this +range for a specific field. + +Here's how you write a node of type `number` in your _searchTree_: +```javascript +{ + key: 'searchNumber', + type: 'number', + field: 'metaScore', + min: 0, + max: 100, +} +``` + +Here is the list of properties that this component expects to have on the node: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `min` | Number | No | Minimum number for the range. | +| `max` | Number | No | Maximum number for the range. | + +**Note:** The properties present in the search tree that aren't used by the node +might be needed for the Provider's type. + +Here's how you write your component: +```javascript +let Number = require('contexture-react/dist/exampleTypes').Number +// ... +// Later, on your render function, or where you put your components: + +``` + +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `NumberInput` | Component | `x => ` | Number input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). +- [(ElasticSearch Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). +- [(MongoDb Provider) Source code of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). +- [(MongoDb Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). +- [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). + +#### ResultCount + + + +The ResultCount component will only show you the number of visible +results compared to the number of total results. It's not an +interactive component. + +Most of your Contexture Trees will have a node with type `results`. +This node posesses information such as the resulting records +themselves, but also which page you're in, how many +elements are per page you will receive, and the total records there +are for this given query, which is what we're looking for for this +type. + +All of these properties are automatically writen by the Contexture +architecture, so we'll be simply passing the tree and the path to the +results type node: + +```javascript +let ResultCount = require('contexture-react/dist/exampleTypes').ResultCount +// ... +// Later, on your render function, or where you put your components: + +``` + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). +- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). +- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). +- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). +- [Source code of the ResultCount component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultCount.js). + +#### ResultPager + + + +The ResultPager component is an interactive component that will show +you which page you're at (in the middle) and what pages are around +your current page, as well as some controllers to move forward and +backwards. + +The style of this component isn't very friendly, but it's very easy to +customize. For more about theme changes, please visit our +[theming docs](../theming/README.md). + +Most of your Contexture Trees will have a node with type `results`. +This node posesses information such as the resulting records +themselves, but also which page you're in, how many +elements are per page you will receive, and the total records there +are for this given query. We use a combination of those values to get +how many pages we can move forward (and backwards), and also to move +around these pages (since our trees react in real time). + +All of these properties are automatically writen by the Contexture +architecture, so we'll be simply passing the tree and the path to the +results type node: + +```javascript +let ResultPager = require('contexture-react/dist/exampleTypes').ResultPager +// ... +// Later, on your render function, or where you put your components: + +``` + +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `List` | Component | `div` | The component that wraps the whole list of pages. | +| `Item` | Component | `span` | The component that wraps each of the available page numbers. | +| `Link` | Component | `a` | An element that wraps each one of the pagination controls. | + +To read more, check the following links: + +- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). +- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). +- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). +- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). +- [Source code of the ResultPager component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultPager.js). + +#### ResultTable + + + +The ResultTable is a component that will display a table with all the +available results, in which each of the result values will be +displayed as columns. + +the results are automatically writen by the contexture architecture, +so we'll be simply passing the tree and the path to the results type +node: + +```javascript +let resulttable = require('contexture-react/dist/exampletypes').resulttable +// ... +// later, on your render function, or where you put your components: + +``` + +This component can be customized by passing any of the following +properties: + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `Table` | Component | `table` | The component that wraps the table list of results. | + +To read more, check the following links: + +- [(ElasticSearch Provider) source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). +- [(elasticSearch Provider) unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). +- [(MongoDb Provider) source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). +- [(MongoDb Provider) unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). +- [Source code of the ResultTable component](https://github.com/smartprocure/contexture-react/blob/master/src/exampletypes/ResultTable.js). ## Other Components ### Generally Useful Components From dffde4d656b32b33af98e5bb7a7e0beb4b387f4f Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:40:57 -0500 Subject: [PATCH 137/150] remove broken imgur link --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index fc292a4..8802d29 100644 --- a/README.md +++ b/README.md @@ -2751,7 +2751,8 @@ To read more, check the following links: #### ResultTable - + +**TODO: ResultTable screenshot** The ResultTable is a component that will display a table with all the available results, in which each of the result values will be From 54e4b855353ffeea43243519edfa053414887e16 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:43:07 -0500 Subject: [PATCH 138/150] add other components section --- README.md | 189 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 189 insertions(+) diff --git a/README.md b/README.md index 8802d29..067a590 100644 --- a/README.md +++ b/README.md @@ -2786,7 +2786,196 @@ To read more, check the following links: ## Other Components ### Generally Useful Components + +#### ContextureProvider + +This component is a magic tool to make Contexture search interfaces without +having to write the tree separated. With this component, you can pass the tree +directly as a plain object, and any children you pass will receive the `tree` +property directly. + +This component receives: + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `types` | Object | Yes | Your client-side types, such as the default types of the Contexture Client. | +| `service` | Object | Yes | The search function that sends the DSL to the initialized Contexture Core. | +| `nodeKey` | Object | Yes | Key for the root node. Defaults with `root`. | +| `...props` | Any other property (children excluded) | Yes | Any other property that you might want to send to the initialization of the Contexture Client. | + +**Note:** Any of the Contexture React Example Types components automatically +add the nodes to the tree if the referenced node is not present. _This is +an experimental feature_. + +Here's how you write your component: +```javascript +let ContextureProvider = require('contexture-react/dist/ContextureProvider') +let ContextureClient = require('contexture-client') +let types = ContextureClient.exampleTypes +let service = async search => ({ + data: await postData('/sarch', { search }) +}) +// ... +// Later, on your render function, or where you put your components: + + + +``` + +- [Source code of the ContextureProvider component](https://github.com/smartprocure/contexture-react/blob/master/src/ContextureProvider.js). + +#### FilterList + +A component that tries to automatically render the specific type components of +the children of a node. + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `node` | Object | Yes | Node of the Contexture Tree where the children that want to be rendered are. | +| `exampleTypes` | Object | No | Object which a key and a component per type. Defaults in the available example types components in contexture-elasticsearch. | +| `fields` | Object | Yes | Object which a key and an object with at least a `label` string property, used to indicate the label of each one of the filters. | + +Here's how you write your component: +```javascript +let FilterList = require('contexture-react/dist/FilterList') +let ContextureClient = require('contexture-client') +let tree = ContextureClient({ + key: 'root', + type: 'group', + schema: 'mySchema', + children: [{ + key: 'query', + type: 'query', + field: 'myFieldName', + query: 'Something' + }] +}) +// ... +// Later, on your render function, or where you put your components: + +``` + +- [Source code of the FilterList component](https://github.com/smartprocure/contexture-react/blob/master/src/FilterList.js). + ### Layout Components +#### Awaiter + +Renders a loading indicator until a Promise is resolved. It will ender +an exception (currently just `Ooops...`) if the Promise fails. if the +Promise passes, the children are rendered. + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `promise` | Promise | Yes | Minumum number for the range. | + +Here's how you write your component: +```javascript +let Awaiter = require('contexture-react/dist/layout/Awaiter') +let promiseMaker = () => new Promise((resolve, reject)) /* ... */ resolve()) +// ... +// Later, on your render function, or where you put your components: + +
My Crazy Children
+ +``` + +- [Source code of the Awaiter component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/Awaiter.js). + +#### BarChart + +Allows you to build your own bar charts (besides just relying on +the DateHistogram component). + +| Property Name | Type | Default Value | Description | +| --- | --- | --- | --- | +| `valueField` | String | `''` | The field of each record where the value used for the height of each barnis located. | +| `categoryField` | String | `''` | The field of each record where the label of each bar is located. | +| `data` | Array (of Objects with both the `valueField` and the `categoryField`) | `[]` | The data that is going to be used to build the chart. | +| `height` | Number | `100` | Specifies the max height of the whole chart. | +| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | +| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | +| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | + +Here's how you write your component: +```javascript +let BarChart = require('contexture-react/dist/layout/BarChart') +// ... +// Later, on your render function, or where you put your components: + +``` + +- [Source code of the BarChart component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/BarChart.js). + +#### SpacedList + +Wraps every children in a div with a given style. Useful for (as the +name portrays) making a spaced list. + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `style` | Object | No | The style that will be applied to each div. Defaults in `{ marginBottom: '25px' }`. | + +Here's how you write your component: +```javascript +let SpacedList = require('contexture-react/dist/layout/SpacedList') +// ... +// Later, on your render function, or where you put your components: + +

Hi

+ + +``` + +- [Source code of the SpacedList component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/SpacedList.js). + +#### TextHighlight + +Used to highlight content within a text basef on a pattern. + +| Property Name | Type | Required | Description | +| --- | --- | --- | --- | +| `pattern` | String | No | RegExp pattern used to find matching content. | +| `text` | String | Yes | Text used as the target of the matches. | +| `Wrap` | Component | No | Component used to wrap each one of the +matched. Defaults to `i`. | + +Here's how you write your component: +```javascript +let TextHighlight = require('contexture-react/dist/layout/TextHighlight') +// ... +// Later, on your render function, or where you put your components: + +``` + +- [Source code of the TextHighlight component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/TextHighlight.js). ## Managing State ### DIY State Management From 6216161ad77fef3c0771ef43750256be62c83447 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:52:36 -0500 Subject: [PATCH 139/150] finish sections --- README.md | 183 ++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 142 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index 067a590..88876c0 100644 --- a/README.md +++ b/README.md @@ -187,12 +187,6 @@ representations of the types that exist on our Contexture Providers. You can read more about Contexture React here, [in the repository](https://github.com/smartprocure/contexture-client). -### Brief History -**TODO** - -### Alternatives & Benchmarks -**TODO** - ## Getting Started #### Install Node 9 and NPM NodeJS 9 and NPM can be installed through [the list of previous @@ -2751,9 +2745,6 @@ To read more, check the following links: #### ResultTable - -**TODO: ResultTable screenshot** - The ResultTable is a component that will display a table with all the available results, in which each of the result values will be displayed as columns. @@ -2977,43 +2968,153 @@ let TextHighlight = require('contexture-react/dist/layout/TextHighlight') - [Source code of the TextHighlight component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/TextHighlight.js). -## Managing State -### DIY State Management -### MobX -### Redux (Coming Soon) - -## Theming - -## Recommendations -### Architecture -#### Client vs Server -#### Type definitions -#### Scaling -### Server Side Searches -#### Searching On an Endpoint -#### Caching -### Client Side Searches -#### Click to Search -#### Real Time Searches -#### Cascading - ## Under the Hood ### Design Principles +- Intentionally stateless. +- Detached from state management tools. +- Can work well with state management tools. +- Modern ES6+ +- Non-strict functional programming. +- Configuration based architecture. +- Focus on a very extensible small core. +- Small DSL. + - Database agnostic. + - Isomorphic Tree State. + - Optimized for database discovery. + - Optimized for advanced search interfaces. + - Aiming to be effective for arbitrarily complex database indexes. + - Simplicity over performance. +- Reaction management. + - Tree walking. + - State flags. + - What updates. + - Pauses. + ### Contexture Core +The core of Contexture is a package of its own. Located at [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture), it offers the very underlying function that is designed to process every one of the search queries. It begins with a simple curried function which, once the providers and the schemas are received, proceeds to process the [Contexture DSL](#TODO) by walking down the given search tree, cleaning up every node of possible inconsistencies, then mutating the tree with the directions given by the provider types and schemas, up until a valid search query is obtained. This query is delivered to the provider `runSearch` method, and the result is finally added back to the tree. + +With this in mind, let's get some specifications. + #### Default Export -#### The Algorithm -#### Utility Functions -### Contexture Providers -#### Contexture ElasticSearch -#### Contexture Mongo -#### Building Your Own Provider +Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. + +The first argument is expected to be a plain JavaScript Object with two keys: + +- `providers`: Should be an object where each key will have a [Contexture Provider](#TODO). +- `schemas`: Should be an object where each key will have a [Contexture Schema](#TODO). + +Calling this function with this object only will return another function, which can be used as an asynchronous search runner. You can also pass in all the arguments as once, but the separation of parameters makes it easier to scope setting up the database providers, the types and the schemas from the search execution. + +Example declaration of a search function by passing the schema & providers object first: + +```javascript +const search = Contexture({ + schemas: { + ...mongoSchemas, + ...elasticSearchSchemas, + }, + providers: { + mongo: require('contexture-mongo')({ /* provider configuration */ }), + elasticsearch: require('contexture-elasticsearch')({ /* provider configuration */}), + }, +}) + +// How you might use it with an express-like API: +// app.use('/search', async req, res) => +// res.send(await search(req.body.search)) +``` + +The other two parameters are the search tree (the Contexture +DSL), and an optional object that will be sent along to the +provider's `runSearch` function as the first parameter, that can +contain any property, but that should at least contain the following +properties: + +| Option | Description | +| ------ | ----------- | +| `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info. | +| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example. | + +This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. + +#### Process Algorithm + +For each of these steps, walk the tree in a parent-first DFS traversal, with each function optionally asynchronous by returning a promise. Along the way, intermediate data is added to contexts on an object called `_meta`. For each context, every type/processor combination is pulled on the fly, meaning it will use the correct local `Provider` and `Type` info even if some contexts have different schemas[^MultiSchema] + +- Clean/Prep everything (adding `_meta`, etc) +- Add `materializedPaths` (used later by `relevantFilters`) +- Run `filter` for each item if it `hasValue` +- Add `relevantFilters` for each item (all filters combined in their groups by the `groupCombinator` except for their own filters and any filters related to them in the tree via an `OR` +- Get `result` for each item if it `hasValidContext` and is not `filterOnly` (as determined by the client event architecture), passing a pre curried search function that includes `relevantFilters` so types don't need to worry about it - logging each request on `_meta.requests` +- Combine `took` values for all requests to get an accurate number and pass results to `onResult` as they come in if they are defined +- Unless in `debug` mode, scrub off `_meta` from the response + +#### Providers +All `Provider` must specify the following properties: + +- `groupCombinator` + - A function that takes the group and an array of its filters and returns them combined. +- `runSearch` + - A function that actually runs the search. It takes the current context, schema, filters (as processed by the `relevantFilters` function and combined with `groupCombinators`), and the criteria for the current context's results (eg `aggs` for an es aggregation or `highlight` for es results). This function can conditionally do different things based on what it is passed - like knowing if it should run an aggregation or a scan/scroll. +- `types` + - A object hash of the `Provider`'s type implementations. It can optionally include a type called `default` whose properties will be used if one of its types are missing something (e.g specifying the default behavior of `validContext` to always allow or prevent results from running) + +Additionally, a provider may expose config for it's client (e.g. `hosts` or request `timeout` for elasticsearch). + +#### Types +All `Types` can implement any if the following properties. All are optional: + +- `filter` + - Takes the current context and produces the filter that will apply to other data contexts in the group (except those related via `OR`). Typically JSON but can be a string as in the SQL case. +- `hasValue` + - Takes the current context and returns a truthy value for whether or not it has a value +- `result` + - Takes the current context, a curried version of the provider's `runSearch` with filters and everything pre-applied (so it is really easy to run searches), the current schema, and the current provider for advanced use cases. This can run one or more async calls - as long as it returns a promise for the final result. If you need to do additional filtering logic, you can use `runSearch` on the provider directly instead of the convenient curried version and inspect the `_meta.relevantFilters` property to see which filters would have been auto-applied, allowing you to do literally any kind of search you want - but there hasn't been a need for this yet. +- `validContext` + - Takes the current context and returns a truthy value for whether or not it should get results. + +[^MultiSchema]: This completely solves and obviates the need for the `MultiIndexGroupProcessor` on the client and handles it in much more elegant way (and in a single service call, instead of `n` services calls). A caveat is that it does not currently handle schemas from different providers (because filters are generated based on their context's local schema), so you can't currently mix a elasticsearch schema with a mongo schema (because it could try to call mongo with elastic search filters for example). + +#### Schemas +Schemas are named by convention based on their filename and should be in `camelCase`. A schema must have one or more provider specific set of configuration properties. + ### Contexture Client -#### State Properties -#### Lenses -#### Reactors in Detail -#### Serialization -#### Traversals -### Contexture React +#### Process Algorithm + +- An action method is called at the top level which: + - Interally mutates the tree + - Makes one or more calls to `dispatch` with relevant event data +- For each dispatched event: + - Validate the entire tree (an async operation) + - Bubble up the tree from the affected node up to the root, and for each node in the path: + - Determine affected nodes by calling the reactor for the current event type + - Mark each affected node for update, or if it is currently paused, mark that it missed updates + - Trigger an update (which is debounced so it does not run right away) +- When the debounce elapses, an update is triggered: + - Check if the update should be blocked + - There may be no affected nodes + - Some nodes might have erros on validation + - Prepare for update - on each node that's markedForUpdate: + - Set the lastUpdateTime to now (to enable dropping stale results later in this process) + - Set `updating` to true + - Serialize the search, omitting all temporary state except lastUpdateTime (which the sever will effectively echo back) and deleting nodes that are filter only with no value + - Execute an actual contexture search + - For each node in the response: + - If the path isn't found in the current tree, ignore it + - If the response is empty, ignore it + - If the response has a lastUpdateTime earlier than the node in the current tree, ignore it (because it's stale) + - If not ignoring the update, mutate the node with the result and set `updating` to false +- After all of this, the promise for the action/dispatch resolves (so you can await the entire process) + +#### Flat Trees + +The client maintains a flat tree in addition to the actual tree, which is an object mapped using `flattenTree` from `futil-js`. +The keys are the array paths encoded as a string, currently using a slashEncoder. +This allows path lookups to perform in constant time at `O(1)`, drastically speeds up some of the internal tree operations. +The paths are also stamped on individual nodes for convenience as performing an action on a node requires knowing its path. + +#### Initialization +On instantiation, the client creates a flat tree representation of the tree and stamps the paths on the nodes. ## Examples - [contexture-site](https://contexture.site) - Showcase of Contexture searches adapting to diverse datasets such as: car crashes, restaurant locations, COVID-19, SAT scores, school grants, bank failures. From ba173db8e4513a1a747d1e36c606bf2e361f19fb Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:54:07 -0500 Subject: [PATCH 140/150] fix small formatting --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 88876c0..65de79b 100644 --- a/README.md +++ b/README.md @@ -2777,7 +2777,6 @@ To read more, check the following links: ## Other Components ### Generally Useful Components - #### ContextureProvider This component is a magic tool to make Contexture search interfaces without From ec53cbf349c00f21bfbafcd0de5d0944fb6b6c95 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:56:01 -0500 Subject: [PATCH 141/150] fix formatting --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 65de79b..8e274c1 100644 --- a/README.md +++ b/README.md @@ -828,7 +828,8 @@ You can read more about these topics in the following links: - [Express Documentation](https://expressjs.com/en/api.html). - [body-parser repository](https://github.com/expressjs/body-parser). - + + 3. Writing a Search Tree Having the DSL processor available through a web server endpoint, we From 04795d98bf04697383819df85eb10ea00076c4ab Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:56:51 -0500 Subject: [PATCH 142/150] fix formatting --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8e274c1..0e01fca 100644 --- a/README.md +++ b/README.md @@ -651,7 +651,8 @@ Please feel free to dig around these topics by following these links: - [Contexture Providers](under-the-hood/contexture-providers/README.md). - [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). - + + 2. The Schemas ```javascript @@ -677,7 +678,8 @@ properties: You can read more about these here: - [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). - + + 3. The MongoDB Client & Our Search Function ```javascript From ca54f86b6d14f65e42bfc816e94abb1d36644c8a Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 21:58:02 -0500 Subject: [PATCH 143/150] fix formatting --- README.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 0e01fca..9fae6a3 100644 --- a/README.md +++ b/README.md @@ -304,7 +304,8 @@ We start by requiring `contexture`. There's nothing much else to see here. ![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) - + + 2. Writing the Schemas ```javascript @@ -498,7 +499,8 @@ Please feel free to dig around these topics by following these links: - [Contexture Providers](under-the-hood/contexture-providers/README.md). - [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). - [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). - + + 2. The ElasticSearch Client ```javascript @@ -527,7 +529,8 @@ being expected by the elasticsearch library. More on the following links: - [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). - + + 3. The Schemas ```javascript @@ -556,7 +559,8 @@ properties: You can read more about these here: - [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). - + + 4. Our Search Function ```javascript From d115896aa72af5718426f6a78a0bfcfa7712bd0e Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 22:00:00 -0500 Subject: [PATCH 144/150] fix glossary --- README.md | 108 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 62 insertions(+), 46 deletions(-) diff --git a/README.md b/README.md index 9fae6a3..392a984 100644 --- a/README.md +++ b/README.md @@ -51,23 +51,27 @@ been doing for almost a decade. ### Glossary of Terms #### Contexture -> NOUN +``` +NOUN -> 1. The fact or manner of being woven or linked together to form a -> connected whole. -> 1.1. A mass of things interwoven together; a fabric. -> 1.2. The putting together of words and sentences in connected -> composition; the construct of a text. -> 1.3. A connected literary structure; a continuous text. +1. The fact or manner of being woven or linked together to form a +connected whole. +1.1. A mass of things interwoven together; a fabric. +1.2. The putting together of words and sentences in connected +composition; the construct of a text. +1.3. A connected literary structure; a continuous text. +``` [Source](https://en.oxforddictionaries.com/definition/us/contexture). #### Domain-Specific Language (DSL) -> A domain-specific language (DSL) is a computer language specialized -> to a particular application domain. This is in contrast to a -> general-purpose language (GPL), which is broadly applicable across -> domains. +``` +A domain-specific language (DSL) is a computer language specialized +to a particular application domain. This is in contrast to a +general-purpose language (GPL), which is broadly applicable across +domains. +``` [Source](https://en.wikipedia.org/wiki/Domain-specific_language). @@ -75,64 +79,76 @@ been doing for almost a decade. According to Wikipedia, `Search Query`, in the context of the `web`, refers to: -> ...a query that a user enters into a web search engine to satisfy his or her -> information needs. Web search queries are distinctive in that they are often -> plain text or hypertext with optional search-directives (such as "and"/"or" -> with "-" to exclude). They vary greatly from standard query languages, which -> are governed by strict syntax rules as command languages with keyword or -> positional parameters. +``` +...a query that a user enters into a web search engine to satisfy his or her +information needs. Web search queries are distinctive in that they are often +plain text or hypertext with optional search-directives (such as "and"/"or" +with "-" to exclude). They vary greatly from standard query languages, which +are governed by strict syntax rules as command languages with keyword or +positional parameters. +``` [Source](https://en.wikipedia.org/wiki/Web_search_query). #### Faceted Search -> Faceted search, also known as faceted navigation or faceted browsing, is a -> technique used by eCommerce brands to help users analyze, organize, and -> filter large sets of product inventory based on filters such as size, color, -> price, and brand. +``` +Faceted search, also known as faceted navigation or faceted browsing, is a +technique used by eCommerce brands to help users analyze, organize, and +filter large sets of product inventory based on filters such as size, color, +price, and brand. +``` [Source](https://www.dynamicyield.com/glossary/faceted-search/). -> Faceted search, also called faceted navigation or faceted browsing, is a -> technique for accessing information organized according to a faceted -> classification system, allowing users to explore a collection of information -> by applying multiple filters. A faceted classification system classifies each -> information element along multiple explicit dimensions, called facets, -> enabling the classifications to be accessed and ordered in multiple ways -> rather than in a single, pre-determined, taxonomic order. +``` +Faceted search, also called faceted navigation or faceted browsing, is a +technique for accessing information organized according to a faceted +classification system, allowing users to explore a collection of information +by applying multiple filters. A faceted classification system classifies each +information element along multiple explicit dimensions, called facets, +enabling the classifications to be accessed and ordered in multiple ways +rather than in a single, pre-determined, taxonomic order. +``` [Source](https://en.wikipedia.org/wiki/Faceted_search). #### Search Filter -> An extension of faceted search, a search filter is a specific product -> attribute a visitor can use to refine the search results of a particular -> category listing, e.g. by size, color, price, or brand. Multiple filters may -> be applied to take a broad range of products and refine them into a more -> narrow selection, allowing the end user to retrieve the most relevant search -> results based on the criteria they’ve selected. +``` +An extension of faceted search, a search filter is a specific product +attribute a visitor can use to refine the search results of a particular +category listing, e.g. by size, color, price, or brand. Multiple filters may +be applied to take a broad range of products and refine them into a more +narrow selection, allowing the end user to retrieve the most relevant search +results based on the criteria they’ve selected. +``` [Source](https://www.dynamicyield.com/glossary/search-filter/). #### Configuration Based Development -> The difference between configuration-driven development and model-driven -> development is that the former is not restricted to the model of the code -> such as classes, fields, and relationships. Configuration-driven development -> (CCD) encompasses anything that can be configured within your application. -> For example, if your architecture dictates that particular business rules -> must be applied consistently across your application, you can use -> configuration files to configure and apply those rules. +``` +The difference between configuration-driven development and model-driven +development is that the former is not restricted to the model of the code +such as classes, fields, and relationships. Configuration-driven development +(CCD) encompasses anything that can be configured within your application. +For example, if your architecture dictates that particular business rules +must be applied consistently across your application, you can use +configuration files to configure and apply those rules. +``` [Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). #### Functional Logic Programming -> Functional logic programming is the combination, in a single programming -> language, of the paradigms of functional programming (including higher-order -> programming) and logic programming (nondeterministic programming, -> unification). This style of programming is embodied by various programming -> languages, including Curry and Mercury. +``` +Functional logic programming is the combination, in a single programming +language, of the paradigms of functional programming (including higher-order +programming) and logic programming (nondeterministic programming, +unification). This style of programming is embodied by various programming +languages, including Curry and Mercury. +``` [Source](https://en.wikipedia.org/wiki/Functional_logic_programming). From 59bb9f17285db3a86c9c82b6e63c521e5aaf2fc4 Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 22:01:02 -0500 Subject: [PATCH 145/150] fix glossary --- README.md | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/README.md b/README.md index 392a984..b96aadf 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,6 @@ been doing for almost a decade. ### Glossary of Terms #### Contexture -``` NOUN 1. The fact or manner of being woven or linked together to form a @@ -60,18 +59,15 @@ connected whole. 1.2. The putting together of words and sentences in connected composition; the construct of a text. 1.3. A connected literary structure; a continuous text. -``` [Source](https://en.oxforddictionaries.com/definition/us/contexture). #### Domain-Specific Language (DSL) -``` A domain-specific language (DSL) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains. -``` [Source](https://en.wikipedia.org/wiki/Domain-specific_language). @@ -79,29 +75,24 @@ domains. According to Wikipedia, `Search Query`, in the context of the `web`, refers to: -``` ...a query that a user enters into a web search engine to satisfy his or her information needs. Web search queries are distinctive in that they are often plain text or hypertext with optional search-directives (such as "and"/"or" with "-" to exclude). They vary greatly from standard query languages, which are governed by strict syntax rules as command languages with keyword or positional parameters. -``` [Source](https://en.wikipedia.org/wiki/Web_search_query). #### Faceted Search -``` Faceted search, also known as faceted navigation or faceted browsing, is a technique used by eCommerce brands to help users analyze, organize, and filter large sets of product inventory based on filters such as size, color, price, and brand. -``` [Source](https://www.dynamicyield.com/glossary/faceted-search/). -``` Faceted search, also called faceted navigation or faceted browsing, is a technique for accessing information organized according to a faceted classification system, allowing users to explore a collection of information @@ -109,26 +100,22 @@ by applying multiple filters. A faceted classification system classifies each information element along multiple explicit dimensions, called facets, enabling the classifications to be accessed and ordered in multiple ways rather than in a single, pre-determined, taxonomic order. -``` [Source](https://en.wikipedia.org/wiki/Faceted_search). #### Search Filter -``` An extension of faceted search, a search filter is a specific product attribute a visitor can use to refine the search results of a particular category listing, e.g. by size, color, price, or brand. Multiple filters may be applied to take a broad range of products and refine them into a more narrow selection, allowing the end user to retrieve the most relevant search results based on the criteria they’ve selected. -``` [Source](https://www.dynamicyield.com/glossary/search-filter/). #### Configuration Based Development -``` The difference between configuration-driven development and model-driven development is that the former is not restricted to the model of the code such as classes, fields, and relationships. Configuration-driven development @@ -136,19 +123,16 @@ such as classes, fields, and relationships. Configuration-driven development For example, if your architecture dictates that particular business rules must be applied consistently across your application, you can use configuration files to configure and apply those rules. -``` [Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). #### Functional Logic Programming -``` Functional logic programming is the combination, in a single programming language, of the paradigms of functional programming (including higher-order programming) and logic programming (nondeterministic programming, unification). This style of programming is embodied by various programming languages, including Curry and Mercury. -``` [Source](https://en.wikipedia.org/wiki/Functional_logic_programming). From 2271ab4b16b5c54de24ce0b047ca614777a6d70d Mon Sep 17 00:00:00 2001 From: ltchris Date: Sun, 8 Mar 2020 22:04:55 -0500 Subject: [PATCH 146/150] fix broken numberings --- README.md | 46 +++++++++++++++++++++++----------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index b96aadf..57ea540 100644 --- a/README.md +++ b/README.md @@ -295,7 +295,7 @@ You can also try this same code in Runkit here: ## What it does -1. Requiring Contexture +Requiring Contexture ```javascript let contexture = require('contexture') @@ -306,7 +306,7 @@ here. ![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) -2. Writing the Schemas +Writing the Schemas ```javascript let schemas = { @@ -325,7 +325,7 @@ schema with a key being `collectionNameSchema`, which is going to be supported by a single provider `mongo`, from which we'll be looking for the `collectionName` collection. -3. Writing the Contexture DSL +Writing the Contexture DSL ```javascript let searchTree = { @@ -354,7 +354,7 @@ that will try to match any record which `name` contains the word `value`. The next node has the `results` type. This is where the results will be written once the search runs. -3. Getting Results +Getting Results ```javascript let result = await contexture({ @@ -399,7 +399,7 @@ Keep in mind that since `contexture` returns a promise, you can change but you'll need to move the code that we have after the await call into the function that is passed on the `.then()` call. -4. Exploring the Results +Exploring the Results ```javascript console.log(result.children[1].context.response.results) @@ -475,7 +475,7 @@ will transform any given query into a working ElasticSearch query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. -1. The Dependencies +The Dependencies ```javascript let Contexture = require('contexture') @@ -501,7 +501,7 @@ Please feel free to dig around these topics by following these links: - [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). -2. The ElasticSearch Client +The ElasticSearch Client ```javascript let elasticClient = null @@ -531,7 +531,7 @@ links: - [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). -3. The Schemas +The Schemas ```javascript let schemas = { @@ -561,7 +561,7 @@ You can read more about these here: - [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). -4. Our Search Function +Our Search Function ```javascript let search = Contexture({ @@ -634,7 +634,7 @@ will transform any given query into a working MongoDB query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. -1. The Dependencies +The Dependencies ```javascript let Contexture = require('contexture') @@ -657,7 +657,7 @@ Please feel free to dig around these topics by following these links: - [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). -2. The Schemas +The Schemas ```javascript let schemas = { @@ -684,7 +684,7 @@ You can read more about these here: - [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). -3. The MongoDB Client & Our Search Function +The MongoDB Client & Our Search Function ```javascript let search = null @@ -747,7 +747,7 @@ the following tasks: Let's dive in. -1. Creating a New Conteture Search Function +Creating a New Conteture Search Function Just as how we saw in the previous pages, creating a new Contexture search function is about setting up the `contexture` package's default @@ -790,7 +790,7 @@ specified the schema's name to be `collectionNameSchema`, and that any search using this schema will be using MongoDb's `collectionName` collection. -2. Create a Web Server With a Search Endpoint +Create a Web Server With a Search Endpoint If we want to separate the direct access to the database from the client, the previous code should live in the server. Our next step is @@ -798,14 +798,14 @@ to expose a `/search` endpoint so our future client can reach this function. In this section, we'll write a simple web server with `express` to satisfy our needs. -2.1. Installing the Dependencies +Installing the Dependencies You'll need to install `express` and `body-parser` at the root of your project, as follows: npm install --save express body-parser -2.2. Writing the Web Server +Writing the Web Server Once you have the dependencies installed, you can set up the server with the following code: @@ -835,8 +835,8 @@ You can read more about these topics in the following links: - [Express Documentation](https://expressjs.com/en/api.html). - [body-parser repository](https://github.com/expressjs/body-parser). - -3. Writing a Search Tree + +Writing a Search Tree Having the DSL processor available through a web server endpoint, we can follow up with the structure of the search interface itself. We'll @@ -867,7 +867,7 @@ Keep in mind that we'll be using `collectionNameSchema` since we already defined a schema with that name on the server's `search.js` file. -4. Make the Search Tree Aware of Contexture +Make the Search Tree Aware of Contexture Having a search tree, we will need `contexture-client` to make it smart enough for the user interface. Let's make sure we have it @@ -916,7 +916,7 @@ You can read more about these topics here: - [MobX](https://mobx.js.org/). - [MobX Observers](https://mobx.js.org/refguide/observer-component.html). -5. Writing a Text Input +Writing a Text Input Having the search tree ready allows us to write a `mobx` observer component that will receive the tree and react to the result changes @@ -1367,7 +1367,7 @@ automatically by invoking one of the following functions: `add`, Let's see some examples: -1. Mutate +Mutate Mutate allows us to change properties on nodes and trigger search afterwards. Here is an example: @@ -1384,7 +1384,7 @@ it is by using our MobX adapter, as shown in our [simple search box](../../getting-started/simple-search-box.md) tutorial, or in greater detail in our [Managing State](managing-state/README.md) docs. -2. Add +Add Having the previous search tree, we can add a children by doing: @@ -1395,7 +1395,7 @@ await searchTree.add(['root'], { }) ``` -3. Remove +Remove We can remove the `results` node we just added by doing: From cc9bf1d014ff9be19f1be557f8f0d81f544b6391 Mon Sep 17 00:00:00 2001 From: me Date: Wed, 11 Mar 2020 04:29:56 +0000 Subject: [PATCH 147/150] GitBook: [master] 30 pages modified --- README.md | 3135 +---------------- SUMMARY.md | 44 + about-contexture/glossary-of-terms.md | 53 + about-contexture/map-of-repos.md | 40 + about-contexture/what-is-contexture.md | 14 + ...connecting-to-elasticsearch-and-mongodb.md | 238 ++ getting-started/project-setup.md | 58 + .../your-first-contexture-script.md | 144 + querying/untitled.md | 2 + types-and-type-components/untitled.md | 2 + untitled/README.md | 72 + untitled/building-your-own-provider.md | 69 + untitled/connecting-other-databases.md | 6 + untitled/contexture-client.md | 4 + untitled/contexture-core.md | 59 + untitled/contexture-dsl.md | 31 + untitled/contexture-elasticsearch.md | 4 + untitled/contexture-mongo.md | 4 + untitled/contexture-react.md | 4 + untitled/design-principles.md | 28 + untitled/discovering-the-database.md | 89 + untitled/diy-types.md | 139 + untitled/elasticsearch-example-types.md | 506 +++ untitled/general.md | 77 + untitled/imdb-example.md | 49 + untitled/layout-components.md | 115 + untitled/mongo-example-types.md | 231 ++ untitled/reactors.md | 24 + untitled/simple-search-box.md | 190 + untitled/your-first-filter.md | 60 + 30 files changed, 2358 insertions(+), 3133 deletions(-) create mode 100644 SUMMARY.md create mode 100644 about-contexture/glossary-of-terms.md create mode 100644 about-contexture/map-of-repos.md create mode 100644 about-contexture/what-is-contexture.md create mode 100644 getting-started/connecting-to-elasticsearch-and-mongodb.md create mode 100644 getting-started/project-setup.md create mode 100644 getting-started/your-first-contexture-script.md create mode 100644 querying/untitled.md create mode 100644 types-and-type-components/untitled.md create mode 100644 untitled/README.md create mode 100644 untitled/building-your-own-provider.md create mode 100644 untitled/connecting-other-databases.md create mode 100644 untitled/contexture-client.md create mode 100644 untitled/contexture-core.md create mode 100644 untitled/contexture-dsl.md create mode 100644 untitled/contexture-elasticsearch.md create mode 100644 untitled/contexture-mongo.md create mode 100644 untitled/contexture-react.md create mode 100644 untitled/design-principles.md create mode 100644 untitled/discovering-the-database.md create mode 100644 untitled/diy-types.md create mode 100644 untitled/elasticsearch-example-types.md create mode 100644 untitled/general.md create mode 100644 untitled/imdb-example.md create mode 100644 untitled/layout-components.md create mode 100644 untitled/mongo-example-types.md create mode 100644 untitled/reactors.md create mode 100644 untitled/simple-search-box.md create mode 100644 untitled/your-first-filter.md diff --git a/README.md b/README.md index 57ea540..802d10c 100644 --- a/README.md +++ b/README.md @@ -1,3135 +1,4 @@ -This is the complete documentation for [Contexture](https://github.com/smartprocure/contexture). +# Contexture Documentation -## About Contexture -### What is Contexture -People of the Internet, here we officialy introduce you to -`contexture`, our framework for building search interfaces. +This is the complete documentation for [Contexture](https://github.com/smartprocure/contexture). -This framework is carefully designed to be a generic solution for a -universe of unlimited possible search interfaces. We've started with a -minimal set of repositories that are representative of tools that -empower our business, but are intended to be merely examples. If -anything, our approaches are only use cases, for the potential of -this tool is ultimately yours to take. - -A quick search over the Internet would reveal that the word -`contexture` means: `the fact or manner of being woven or linked -together to form a connected whole` and also `the putting together of -words and sentences in connected composition; the construction of a -text`. - -Picking `contexture` as the name for this project means that we are -trying to expose not only our ultimate intentions, but also more or -less how the system is built. The way our projects work is by a DSL -that is used to gather different intended search inputs, each one -representing some useful abstraction of a search filter (like an input -where you can write a word to be searched, or another where you can -filter the search results by one or more options), then using the -values to process a DSL that will end up retrieving values from one or -more different databases, then returning these values on the -respective sections of the DSL, so that each result can update each -one of the components of the user interface. A more detailed -description is visible in the following diagram. - -

- -The canonical example of a Contexture Node is faceted search, where -you have a checkbox list that is both a filter (in the sense that it -restricts results based on the checked values) and an aggregation -(which shows the top n values that can be checked). Contexture allows -them to be nested in advanced searches with boolean joins like -`and`/`or`/`not`. - -

- -This thought process will become more clear as we progress through the -docs. Hopefully, some pages later it will be easy to grasp how we -provide a new perspective on building search interfaces, and perhaps -even how you can use it to power up your business, just like we have -been doing for almost a decade. - -### Glossary of Terms -#### Contexture - -NOUN - -1. The fact or manner of being woven or linked together to form a -connected whole. -1.1. A mass of things interwoven together; a fabric. -1.2. The putting together of words and sentences in connected -composition; the construct of a text. -1.3. A connected literary structure; a continuous text. - -[Source](https://en.oxforddictionaries.com/definition/us/contexture). - -#### Domain-Specific Language (DSL) - -A domain-specific language (DSL) is a computer language specialized -to a particular application domain. This is in contrast to a -general-purpose language (GPL), which is broadly applicable across -domains. - -[Source](https://en.wikipedia.org/wiki/Domain-specific_language). - -#### Search Query - -According to Wikipedia, `Search Query`, in the context of the `web`, refers to: - -...a query that a user enters into a web search engine to satisfy his or her -information needs. Web search queries are distinctive in that they are often -plain text or hypertext with optional search-directives (such as "and"/"or" -with "-" to exclude). They vary greatly from standard query languages, which -are governed by strict syntax rules as command languages with keyword or -positional parameters. - -[Source](https://en.wikipedia.org/wiki/Web_search_query). - -#### Faceted Search - -Faceted search, also known as faceted navigation or faceted browsing, is a -technique used by eCommerce brands to help users analyze, organize, and -filter large sets of product inventory based on filters such as size, color, -price, and brand. - -[Source](https://www.dynamicyield.com/glossary/faceted-search/). - -Faceted search, also called faceted navigation or faceted browsing, is a -technique for accessing information organized according to a faceted -classification system, allowing users to explore a collection of information -by applying multiple filters. A faceted classification system classifies each -information element along multiple explicit dimensions, called facets, -enabling the classifications to be accessed and ordered in multiple ways -rather than in a single, pre-determined, taxonomic order. - -[Source](https://en.wikipedia.org/wiki/Faceted_search). - -#### Search Filter - -An extension of faceted search, a search filter is a specific product -attribute a visitor can use to refine the search results of a particular -category listing, e.g. by size, color, price, or brand. Multiple filters may -be applied to take a broad range of products and refine them into a more -narrow selection, allowing the end user to retrieve the most relevant search -results based on the criteria they’ve selected. - -[Source](https://www.dynamicyield.com/glossary/search-filter/). - -#### Configuration Based Development - -The difference between configuration-driven development and model-driven -development is that the former is not restricted to the model of the code -such as classes, fields, and relationships. Configuration-driven development -(CCD) encompasses anything that can be configured within your application. -For example, if your architecture dictates that particular business rules -must be applied consistently across your application, you can use -configuration files to configure and apply those rules. - -[Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). - -#### Functional Logic Programming - -Functional logic programming is the combination, in a single programming -language, of the paradigms of functional programming (including higher-order -programming) and logic programming (nondeterministic programming, -unification). This style of programming is embodied by various programming -languages, including Curry and Mercury. - -[Source](https://en.wikipedia.org/wiki/Functional_logic_programming). - -### Map of Repos - -The Contexture framework comes to life through a list of repositories -that individually specialize in some needed layer for our -architecture. We will be using most (if not all) of these projects in -our upcoming pages. Let's explore the repos we have so far: - -#### Contexture Core - -[github.com/smartprocure/contexture](https://github.com/smartprocure/contexture) -is where our main DSL processor lives. It is a very small layer that -ties everything together. This one receives the information about the -different search representations, about the databases involved, and -the DSL, then outputs the search results respective to each one of the -queries described in a copy of the received DSL. - -You can read more about the core here, [in the repository](https://github.com/smartprocure/contexture). - -#### Contexture Providers - -The Contexture Providers are the interfacing layer that ties the -Contexture DSL to the targeted databases. So far, we have only two -open source providers: - -- [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) -- [contexture-mongo](https://github.com/smartprocure/contexture-mongo) - -If you are planning to use either ElasticSearch or MongoDB for your -project, the best way to get started is to use those repositories. -However, if you need to use Contexture with another database, you will -need to implement the provider yourself. We're looking for code -contributors, so please don't feel limited to the current available -tools. Help us grow together! - -#### Contexture Client - -The Contexture Client is responsible for triggering behaviors on the -search interfaces by knowing what causes changes in one or more -elements of the search tree. It is the key piece of technology that -allows our search interfaces to work in real time. - -You can read more about the Contexture Client here, [in the repository](https://github.com/smartprocure/contexture-client). - -#### Contexture React - -The Contexture React repository holds a list of components that -facilitate building search interfaces. They are mainly graphical -representations of the types that exist on our Contexture Providers. - -You can read more about Contexture React here, [in the repository](https://github.com/smartprocure/contexture-client). - -## Getting Started -#### Install Node 9 and NPM -NodeJS 9 and NPM can be installed through [the list of previous -releases of NodeJS](https://nodejs.org/en/download/releases/). You -might get it working with Node 10 (if so, let us know). We haven't -fully upgraded to Node 10 yet, so until then, we encourage you to at -least have a working version of Node 9 in hand. - -An easy way to move from one version to another is with -[nvm](https://github.com/creationix/nvm). Here's a command you can run -to install nvm: - - `curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash` - -Please follow [nvm's README](https://github.com/creationix/nvm/blob/master/README.md) for more information. - -#### Install Contexture -Once you have NodeJS and NPM installed, you'll need either a new -folder for your new project with Contexture, or to go to an existing -project, then run: - - npm install --save contexture - -#### Install Contexture Client -To install the `contexture-client` you can run the following -command in your project's root folder: - - npm install --save contexture-client - -#### Install Contexture ElasticSearch -To install the `contexture-elasticsearch` you can also run the -following command in your project's root folder: - - npm install --save contexture-elasticsearch - -#### Install Contexture Mongo -To install the `contexture-mongo` you can also run the following -command in your project's root folder: - - npm install --save contexture-mongo - -#### Install Contexture React -To install the `contexture-react` you can also run the following -command in your project's root folder: - - npm install --save contexture-react - -### A First Contexture Script -With everything installed, let's see a simple script that runs a -simple one-time search: - -```javascript -let contexture = require('contexture') - -let schemas = { - collectionNameSchema: { - mongo: { - collection: 'collectionName' - } - } -} - -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'results', - type: 'results' - }] -} - -let result = await contexture({ - schemas, - providers: { - mongo: require('contexture-mongo')({ - types: require('contexture-mongo/types')(), - getClient: () => ({ - // Fake mongo client. - // For this example we only care about - // collection().aggregate([...]).toArray() being a promise. - collection: name => ({ - aggregate: aggregations => ({ - toArray: async () => ['Unrealistic result example'] - }) - }) - }) - }) - }, -}, searchTree) - -console.log(result.children[1].context.response.results) - -// Explore it yourself! -result -``` - -You can also try this same code in Runkit here: - - -## What it does - -Requiring Contexture - -```javascript -let contexture = require('contexture') -``` -We start by requiring `contexture`. There's nothing much else to see -here. - -![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) - - -Writing the Schemas - -```javascript -let schemas = { - collectionNameSchema: { - mongo: { - collection: 'collectionName' - } - } -} -``` -Immediatly afterwards, we define our schemas. For this specific -example, we are going to emulate doing a search on a single collection -of a Mongo database. We define `collectionName` as the name of this -arbitrary collection. The `schemas` object ends up containing a single -schema with a key being `collectionNameSchema`, which is going to be -supported by a single provider `mongo`, from which we'll be looking -for the `collectionName` collection. - -Writing the Contexture DSL - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'results', - type: 'results' - }] -} -``` -Our next step is to define a simple search query using Contexture DSL. -This search query will have a mandatory root group, which among other -features indicates which schema we will be running this query on -(`collectionNameSchema`). This node will have two children. The first -children is going to be our search query. This query will be a `text` -type query. We're indicating that we want to run a plain text search -that will try to match any record which `name` contains the word -`value`. The next node has the `results` type. This is where the -results will be written once the search runs. - -Getting Results - -```javascript -let result = await contexture({ - schemas, - providers: { - mongo: require('contexture-mongo')({ - types: require('contexture-mongo/types')(), - getClient: () => ({ - // Fake mongo client. - // For this example we only care about - // collection().aggregate([...]).toArray() being a promise. - collection: name => ({ - aggregate: aggregations => ({ - toArray: async () => ['Unrealistic result example'] - }) - }) - }) - }) - }, -}, searchTree) -``` - -The next thing we do is to actually run a one-time search. In this -case, we `await` for `contexture`, passing along some very important -parameters. First, we pass the schemas we previously defined. Then, we -pass a providers object, which has the provider for `mongo`. This -`mongo` key will be matched against the `mongo` key that we defined in -the schemas, so you could change this property name to something -entirely different. When we send the mongo provider, we assign the -result of the initialization of `contexture-mongo`, where we send the -`contxture-mongo/types` (which **needs to be called as a function once -required**) and a `getClient` function. In a real schenario, you would -just send the object that results of `require('mongodb')`. However, to -provide an exceutable example in the browser, we've made a very small -Mock of MongoDB where we will return a same object for any collection -call, which will only allow fake aggregations, which will have a -promise `toArray` function that will return our `Unrealistic result -example`. - -Keep in mind that since `contexture` returns a promise, you can change -`await contexture({ /* .. */ })` to `contexture({ /* .. */ }).then()`, -but you'll need to move the code that we have after the await call -into the function that is passed on the `.then()` call. - -Exploring the Results - -```javascript -console.log(result.children[1].context.response.results) - -// Explore it yourself! -result -``` - -Finally, we can use our search result! The output of the search will -be added to a copy of the original search query. The location of this -result will be in the children of the `root` node that has the -`results` type, within a `context` object, within a `response` object, -on a `results` property. This whole object is going to be exposed in -the _runkit_ example for you to play with it. - -### Connecting to Elasticsearch & MongoDB -Our primary use case for advanced search interfaces is to query data -that we store on ElasticSearch and MongoDB. Because of that, we -provide two contexture libraries for those databases. In this page -we'll examine how to use these repositories to connect to existing -Mongo databases and ElasticSearch indexes. - -#### Connecting to ElasticSearch -The followng code example shows how to connect to ElasticSearch: - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-elasticsearch') -let types = require('contexture-elasticsearch/types') -let elasticsearch = require('elasticsearch') -let AgentKeepAlive = require('agentkeepalive') - -let elasticClient = null -let getClient = () => { - if (elasticClient) return elasticClient - elasticClient = elasticsearch.Client({ - minSockets: 1, - maxSockets: 20, - keepAlive: true, - createNodeAgent: (connection, config) => - new AgentKeepAlive(connection.makeAgentConfig(config)) - }) - return elasticClient -} - -let schemas = { - yourCustomSchemaName: { - elasticsearch: { - index: 'SomeIndex', - type: 'SomeType' - } - } -} - -let search = Contexture({ - schemas, - providers: { - elasticsearch: provider({ - getClient, - request: { - headers: { - 'custom-header-app-name': 'my-app-sent-this' - } - }, - types: types() - }) - } -}) -``` - -The code above will provide a working search function `search` that -will transform any given query into a working ElasticSearch query, -which will be sent to the database to retrieve the data. Let's examine -this code in greater detail. - -The Dependencies - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-elasticsearch') -let types = require('contexture-elasticsearch/types') -let elasticsearch = require('elasticsearch') -let AgentKeepAlive = require('agentkeepalive') -``` - -The first six lines are about requiring dependencies, the first -dependency being just `contexture`. The second one is our database -provider, `contexture-elasticsearch`. Then, we require the types; even -though you will probably have to write your own types, we have some -pre-defined types available at `contexture-elasticsearch/types`. -Lastly, we require `elasticsearch` and `agentkeepalive`, so we can -actually connect to ElasticSearch and keep it connected even if the -address becomes unreachable for a while. - -Please feel free to dig around these topics by following these links: - -- [Contexture Providers](under-the-hood/contexture-providers/README.md). -- [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). -- [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). - - -The ElasticSearch Client - -```javascript -let elasticClient = null -let getClient = () => { - if (elasticClient) return elasticClient - elasticClient = elasticsearch.Client({ - minSockets: 1, - maxSockets: 20, - keepAlive: true, - createNodeAgent: (connection, config) => - new AgentKeepAlive(connection.makeAgentConfig(config)) - }) - return elasticClient -} -``` - -Now, we define a utility function to connect to ElasticSearch just -once. The idea here is to be able to re-use the same connection -instead of using new clients every time a search is executed. We do -this by declaring a function that will return `elasticClient` if it has -a truthy value, or otherwise instantiate a new elasticsearch client -with the given configuration. The configuration we are sending is -merely an example and shouldn't be used without understanding what is -being expected by the elasticsearch library. More on the following -links: - -- [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). - - -The Schemas - -```javascript -let schemas = { - elasticsearch: { - index: 'SomeIndex', - type: 'SomeType' - } -} -``` - -As shown above, the next thing we do is to define the schemas. This is -an object which properties are the names of each one of the schemas -that will be available for the contexture DSL. The schemas for the -ElasticSearch provider can specify any or all of the following -properties: - -| Option | Type | Description | Required | -| ------ | ---- | ----------- | -------- | -| `index` | `string` | Which ES index to use when querying | x | -| `type` | `string` | Which ES type to use when querying | | -| `summaryView` | `function` | Used by `results` to return a summary view instead of the whole document, (e.g. for indexes with many fields). Defaults to returning the `hit` property. | | -| `highlight` | `object` | Used by `results` to determine what fields to highlight, and whether or not they are `inline` (copied over inline on to the source) or `additional` (in a list of additional fields that matched) | | -| `forceExclude` | `array` | Used by `results` to extend the exclude fields provided on the search tree. The extension happens only if the results node has a `forceExclude` flag set to true. - -You can read more about these here: - -- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). - - -Our Search Function - -```javascript -let search = Contexture({ - schemas, - providers: { - elasticsearch: provider({ - getClient, - request: { - headers: { - 'custom-header-app-name': 'my-app-sent-this' - } - }, - types: types() - }) - } -}) -``` - -Once we have the schemas set, we can create our `search` function. -For this purpose, we will be calling `Contexture` with just one -object. This object will have the `schemas`, and the `providers`. The -providers will host just one key/value, the one specific for -`elasticsearch`. The provider, which we required at the beginning of -the script, needs to be called with the `getClient` function we just -created and the `types()` that we got from the -`contexture-elasticsearch/types` repository. We also show that you can -customize the request headers by providing an object that includes the -headers keys and values. This `search` function is ready to receive -search trees and write the results back! - -This example and many other important details about -`contexture-elasticsearch` are accessible in the following links: - -- [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). - -#### Connecting to MongoDB -The followng code example shows how to connect to MongoDB: - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-mongo') -let types = require('contexture-mongo/types') -let MongoClient = require('mongodb').MongoClient - -let schemas = { - yourCustomSchemaName: { - mongo: { - collection: 'SomeCollection' - } - } -} - -let search = null - -MongoClient.connect('mongodb://localhost:27017', function(err, client) { - search = Contexture({ - schemas, - providers: { - mongo: provider({ - getClient: () => client, - types: types() - }) - } - }) -}) -``` - -The code above will provide a working search function `search` that -will transform any given query into a working MongoDB query, -which will be sent to the database to retrieve the data. Let's examine -this code in greater detail. - -The Dependencies - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-mongo') -let types = require('contexture-mongo/types') -let MongoClient = require('mongodb').MongoClient -``` - -The first lines are about requiring dependencies, the first -dependency being just `contexture`. The second one is our database -provider, `contexture-mongo`. Then, we require the types; even -though you will probably have to write your own types, we have some -pre-defined types available at `contexture-mongo/types`. -Lastly, we require `mongodb` to get the much needed `MongoClient`, so -we can actually connect to the database. - -Please feel free to dig around these topics by following these links: - -- [Contexture Providers](under-the-hood/contexture-providers/README.md). -- [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). - - -The Schemas - -```javascript -let schemas = { - yourCustomSchemaName: { - mongo: { - collection: 'SomeCollection' - } - } -} -``` - -As shown above, the next thing we do is to define the schemas. This is -an object which properties are the names of each one of the schemas -that will be available for the contexture DSL. The schemas for the -MongoDB provider can specify any or all of the following -properties: - -| Option | Type | Description | Required | -| ------ | ---- | ----------- | -------- | -| `collection` | `string` | The MongoDB collection that will be used to run the queries | x | - -You can read more about these here: - -- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). - - -The MongoDB Client & Our Search Function - -```javascript -let search = null - -MongoClient.connect('mongodb://localhost:27017', function(err, client) { - search = Contexture({ - schemas, - providers: { - mongo: provider({ - getClient: () => client, - types: types() - }) - } - }) -}) -``` - -Next we will need to connect to MongoDB. In the previous code, we -provide an example in which we connect to `mongodb://localhost:27017`, -and using a callback to the `connect` method, we are able to obtain -the database client that we need. Once we have this client, we can -actually create our `search` function. For this purpose, we will -be calling `Contexture` with just one object. This object will have -the `schemas`, and the `providers`. The providers will host just one -key/value, the one specific for `mongo`. The provider, which -we required at the beginning of the script, needs to be called with -a `getClient` function that will just return the database client, and -the `types()` that we got from the `contexture-mongo/types` repository. -With these steps completed, we end up with a function that is ready to -receive search trees and write the results back! - -You can read more about these topics in the following links: - -- [Connecting to MongoDB using the native MongoDB driver for NodeJS](http://mongodb.github.io/node-mongodb-native/api-articles/nodekoarticle1.html#getting-that-connection-to-the-database). - -This example and many other important details about -`contexture-mongo` are accessible in the following links: - -- [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). - -### Connecting to Other Databases -Contexture depends on it's providers to be able to know how to -translate from the Contexture DSL to the specific DSL that each -database needs. Because of this, to connect to other databases you -will need to create a new Provider. - -### Simple Search Box -Building a simple search box consists of a single input field tied to -any of the fields that any record might have in the specified database -or index. To be able to make this text input tied to the search -structure, we will need to bring `contexture-client` in. With that in -mind, and the knowledge we already have of contexture, we can define -the following tasks: - -1. Create a New Contexture Search Function. -2. Create a Web Server With a Search Endpoint -3. Write a Search Tree. -4. Make the Search Tree Aware of Contexture. -5. Write a Text Input. - -Let's dive in. - -Creating a New Conteture Search Function - -Just as how we saw in the previous pages, creating a new Contexture -search function is about setting up the `contexture` package's default -export with `schemas` and `providers`. In this case, we'll use the -contexture-mongo approach in the following file (let's call it -`search.js`): - -```javascript -let Contexture = require('contexture') -let provider = require('contexture-mongo') -let types = require('contexture-mongo/types') -let MongoClient = require('mongodb').MongoClient - -let schemas = { - collectionNameSchema: { - mongo: { - collection: 'collectionName' - } - } -} - -module.exports = {} - -MongoClient.connect('mongodb://localhost:27017', function(err, client) { - module.exports.search = Contexture({ - schemas, - providers: { - mongo: provider({ - getClient: () => client, - types: types() - }) - } - }) -}) -``` - -Note that we're now exporting the search function with -`module.exports.search = Contexture(...`. You can also note that we -specified the schema's name to be `collectionNameSchema`, and that any -search using this schema will be using MongoDb's `collectionName` -collection. - -Create a Web Server With a Search Endpoint - -If we want to separate the direct access to the database from the -client, the previous code should live in the server. Our next step is -to expose a `/search` endpoint so our future client can reach this -function. In this section, we'll write a simple web server with -`express` to satisfy our needs. - -Installing the Dependencies - -You'll need to install `express` and `body-parser` at the root of your -project, as follows: - - npm install --save express body-parser - -Writing the Web Server - -Once you have the dependencies installed, you can set up the server -with the following code: - -```javascript -let express = require('express') -let bodyParser = require('body-parser') -let search = require('./search') -let app = express() - -// create application/json parser -let jsonParser = bodyParser.json() - -app.post('/search', jsonParser, (req, res) => { - if (!req.body || !req.body.search) return res.sendStatus(400) - search(req.body.search).then((err, result) => { - if (err) return res.send(401, err) - res.send(200, result) - }) -}) - -app.listen(3000) -``` - -You can read more about these topics in the following links: - -- [Express Documentation](https://expressjs.com/en/api.html). -- [body-parser repository](https://github.com/expressjs/body-parser). - - -Writing a Search Tree - -Having the DSL processor available through a web server endpoint, we -can follow up with the structure of the search interface itself. We'll -conceptualize this by writing the Contexture DSL itself. - -Let's use the same `searchTree` that we used in our first -script: - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'results', - type: 'results' - }] -} -``` - -Keep in mind that we'll be using `collectionNameSchema` since we -already defined a schema with that name on the server's `search.js` -file. - -Make the Search Tree Aware of Contexture - -Having a search tree, we will need `contexture-client` to make it -smart enough for the user interface. Let's make sure we have it -available in our project with: - - npm install --save contexture-client - -We will also use MobX to make it easier to notify our component that -the state has changed. For that purpose, we will need to install -`mobx` using NPM: - - npm install --save mobx - -Since we're heavy users of MobX, `contexture-client` already -provides an adapter that we can use out of the box. Knowing this, -let's prepare our `contexture-client` to work well with -`mobx`, as follows: - - -```javascript -let ContextureClient = require('contexture-client') -let ContextureMobx = require('contexture-react/dist/utils/contexture-mobx') - -let types = ContextureClient.exampleTypes -let service = async search => ({ - data: await postData('/sarch', { search }) -}) - -let Contexture = ContextureMobx({ - types, - service, -}) -``` - -Note that our service function will be the one responsible for sending -the `searchTree` we previously defined to the DSL processor, we can -wrap the search tree into a smart object that will later react to both -the user input, and the search results. - -```javascript -let contextureSearchTree = Contexture(searchTree) -``` - -You can read more about these topics here: - -- [MobX](https://mobx.js.org/). -- [MobX Observers](https://mobx.js.org/refguide/observer-component.html). - -Writing a Text Input - -Having the search tree ready allows us to write a `mobx` observer -component that will receive the tree and react to the result changes -immediately. Here's an example: - -``` -let { observer } = require('mobx') -let SearchQuery = observer(({ tree }) => - { - tree.getNode(['root', 'namequery']).value = e.target.value - }} - /> - -let SearchResults = observer(({ tree }) => ( -
- {JSON.stringify(tree.getNode(['root', 'results']).context.response.results)} -
-)) -``` - -Which we would render this way: - -```javascript - - -``` - -This will generate a text input that will trigger a new search every -time the contents of the input change. This new search will get the -results and show the results in a JSON form (because of the -`JSON.stringify` part). Ideally, you will not render them in a JSON -form, but render them using a list component or table. - -### Your First Filter -In the previous page, we wrote a simple search user interface where a -single input would retrieve results. Now, we will add a simple filter -that will allow us to get results within a given range. - -#### Adding the Filter to the Tree - -The way we will add the filter is by adding a node to the tree. This -node is going to have a type of `number`, which asks for a `field`, a -`min` and a `max` values. - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'numberrange', - type: 'number', - field: 'age', - min: 0, - max: 100 - }, { - key: 'results', - type: 'results' - }] -} -``` - -With the `numberrange` node added, we can create another component -with two inputs that will allow us to filter results with the range -that the user specifies: - -```javascript -let RangeComponent = observer(({ tree }) => ( -
- Minimum: - { - tree.getNode(['root', 'numberrange']).min = e.target.min - }} - /> -
- Maximum: - { - tree.getNode(['root', 'numberrange']).max = e.target.max - }} - /> -
-)) -``` - -And that's it! Rendering this component will make our search aware of -any change the user might desire on the minimum and maximum ages they -might want to use to filter the available results. - -### Discovering the Database - -One of the great things about Contexture is that it can be used to -discover the database. In this page, we'll see how to write a filter -that not only allows the user to refine their search, but that also -shows information about our data that might not be obvious by looking -at the results directly. - -### The Facet Filter - -The `facet` type is just like any other type in the tree. It requires -a unique key and some other properties. In contrast to previously seen -types, such as `text` and `number`, facet doesn't require specific -values, but instead it asks for two properties: `field`, and -optionally `size`. This type is incredibly useful because besides -filtering the results based on the `value` (or `values`) the user -might have chosen, it retrieves from the database a list of available -values! The `facet` type can be very well represented as a list of -checkboxes, ready for users to pick for one or more values. Let's see -how we can use it. - -#### Adding the Facet Filter to the Tree - -To add the `facet` filter to the tree, we simply add it to the -structure we already had. For example: - -```javascript -let searchTree = { - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }, { - key: 'numberrange', - type: 'number', - field: 'age', - min: 0, - max: 100 - }, { - key: 'cityfacet', - type: 'facet', - field: 'city', - value: 'Orlando', // Optional - // We could have instead `values` with ['Orlando', 'Miami'] - }, { - key: 'results', - type: 'results' - }] -} -``` - -Once we have it in the tree, we can simply create another component to -allow users to set a value to this type. In this case, however, we -will be retrieving the available values from a field that is -automatically inserted to this part of the tree. Let's make sure we -have the data before we render the components. - -#### Fetching the Available Options - -Having the facet node already added to the tree, it's only matter of -running a first search before we render the components to actually get -the possible results from the `facet` type. We can achieve this by -calling to `contexture-client`'s `refresh` function: - -```javascript -// This would go after: -// let contextureSearchTree = Contexture(searchTree) -contextureSearchTree.refresh(['root']) -``` - -Now, the available options will be ready to be used at: -`contextureSearchTree.getNode(['root', 'cityfacet']).context.options`. -So we can write our facet component this way: - -```javascript -let toggleValue = (array, value) => { - if (array.indexOf(value) > -1) { - let copy = array.slice() // Making sure it's not a MobX Array - copy.splice(array.indexOf(value), 1) // removing value from the array - array.replace(copy) // MobX Arrays have this method to replace the array's inner values - } else { - array.push(value) - } -} -let RangeComponent = observer(({ tree }) => ( -
- Select One or More: - {tree.getNode(['root', 'cityfacet']).context.options.map(option => ( -
- -1} - onChange={() => toggleValue(tree.getNode(['root', 'cityfacet']).values, option.value)} - /> - -
- ))} -
-)) -``` - -Including this component in our search interface will show the current -available cities for the search results we have so far, and will allow -users to filter the search even more by letting them pick any (or many) -of the available cities. - -### IMDB Index - -In our Contexture React repository, we've created a Storybook live -example that uses Contexture (with no server whatsoever, everything in -the client) to query and discover a public ElasticSearch index with -data similar to the data that IMDB might contain. - -- [Contexture React Repository](https://github.com/smartprocure/contexture-react). -- More information about [Storybook](https://github.com/storybooks/storybook). -- [The IMDB Live Index Explorer with Contexture](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). -- [The IMDB Live Index Explorer Source Code](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). - -Let's see how we can use this tool. - -#### Picking a Field - -The tool initially shows you a button named `Pick a Field`: - - - -This button will allow you to select any field on which you might want -to run a search. If you click it, you'll see the following fields: - - - -If we pick `Actors`, for example, the search will trigger and results -will appear: - - - -Now, let's click on `Select Type` and click on `Facet`. A moment -afterwards, we will get a list of checkboxes with the most common -actors among the results: - - - -We can see that the actor that has made the most movies (among the -ones idexed) is Naveen Andrews. We can also see the number of movies -this and other actors have made, and the total actors in the database -(of which we can only see 10 in this screenshot). Under this list of -actors, you'll see a text saying `View More`. If you click it, the -next 10 actors will appear (in order). - -At the bottom of the page, you'll see a button labeled `Add Filter`. -Clicking it will add another interactive component to the website, -which will say `Click to add AND`. - - - -Clicking the `Click to add AND` button will show us again a `Pick a -Field` button: - - - -So we can start again. Let's say we pick the `Genre` field, and we -click for the `Facet` type again. A bit later, the list of ordered -genres will appear: - - - -And that's it! We're discovering the database with a very simple and -unpolished interface. By this point you might be curious on what -components we're using to do all this magic. There's really no trick, -just follow us through the tutorial. You can also skip some steps if -you're particularly interested in: - -- [Contexture React Repository](https://github.com/smartprocure/contexture-react). - -## Querying -### Contexture DSL -The Contexture DSL is a JavaScript object structure, or JSON -structure, that is composed of nested nodes that are equal at all -levels. Each node has a `key`, a `type` and many other optional -properties. The first node is called the root and is a node of type -`group`. Any node of type `group` can have many children. Each -children can be a node of any type. If any children is another group, -this children will probably have one or more other children nodes of -any type, and so on. - -Let's begin talking about the root. - -#### The Root - -The root of a Contexture Tree is a node of type `group`. Group types -are required to have a `key`, the `type: "group"` property, and an -array of children `children: [/* nodes */]`. Root groups also need an -extra property: then ame of the schema. In summary, this is how a root -node should look: - -```javascript -let searchTree = { - key: 'myRootNode', - type: 'group', - schema: 'mySchemaName', - children: [ - // Other nodes - ] -} -``` - -#### The Children -Each children will be an individual node. Each node will have at least -a unique `key` (unique per tree, but they can appear again in other -trees), and a `type`. Some types might require more properties. - -That's really all that it is for our DSL. The magic happens in the -types and the providers. As long as your types are defined properly, -and your providers build the correct queries and write them back in -the tree, Contexture will work and you'll be able to use the -Contexture DSL to build search interfaces of any complexity. - -**Note:** Many of the types have some common properties, like `field` -or `values`. The only reason these properties are common is because -they've made sense to be common for each one of our specific types, -not because they have something to share between types. Each type has -it's own properties and rules, so you should treat them as independent -structures. - -### Interactive Queries -#### Contexture Client -Having a Contexture DSL by itself won't work since it's only a -language and not an interactive or working program in any sense. We -need something to automatically send this query to our DSL processor -(the main `contexture` repository). For this purpose, we provide the -`contexture-client`. - -The Contexture Client is responsible for keeping track of the changes -that happen on the Contexture DSL, either by the user or by the -Contexture core. The nodes in the Contexture DSL host values used for -filtering the results, but also hold other properties, such as the -search results, and a `_meta` object that you can use for debugging -purposes. In this page, we will sumarize how the Contexture Client -works with the Contexture DSL to provide real-time interactive -searches. - -##### Initializing the Contexture Client -The Contexture Client has a default export that needs to be called -first with some configuration properties before being able to process -any search. This is how you would normally initialize it: - -```javascript -let ContextureClient = require('contexture-client') -let Contexture = ContextureClient({ - service, - types, -}) -``` - -When Contexture Client determines the search query is ready to be -executed, it will call the `service` function provided with the whole -Contexture DSL. It expects to receive another full DSL that it parses -to then change the nodes again. As previously mentioned, an example -`service` would be: - -```javascript -let service = async search => ({ - data: await postData('/sarch', { search }) -}) -``` - -The `types` property is an object where each key is a type's name. -These type definitions are different than the ones from the servers -and allow our Contexture Client to do three things: - -- To know how to validate a node. -- To know what happens to a specific node (and it's suroundings) when - it changes. -- To know how to complemente any missing value on a specific node - (default values, so you can omit properties when you write each of - the nodes). - -Each one of the types will have the following properties: - -| Property Name | Type | Description | -| --- | --- | --- | -| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | -| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | -| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | - -##### Context Tree -Once you have Contexture Client initialized, we will be able to pass -the Contexture DSL into it. This will generate something we call the -`Context Tree`, which is an object with the following properties: - -| Name | Signature | Description | -| ---- | --------- | ----------- | -| add | `async (path, newNode) -> await searchCompleted` | Adds a node to the tree as a child of the specified path. You can await this for when updates settle and relevant searches are completed. | -| remove | `async path -> await searchCompleted` | Removes a node at the specified path. You can await this for when updates settle and relevant searches are completed. | -| mutate | `async (path, deltas) -> await searchCompleted` | Mutates the node at the given path with the new values. You can await this for when updates settle and relevant searches are completed. | -| getNode | `[path] -> node` | Lookup a node by a path (array of keys). | -| tree | tree | A reference to the internal tree. If you mutate this, you should dispatch an appropriate event. | - -You can read more about the instantiated client here: [contexture-client Run Time](https://github.com/smartprocure/contexture-client#run-time). - -##### Tree and getNode -Context Trees have two properties for nagivating through the -Contexture DSL: `tree` and `getNode`. The `tree` property will have -the DSL as it was received, plus the default properties set by each -one of the client types, and some state properties. On the other hand, -`getNode` is a quick way to access the tree by sending only the keys -of each node. For example, let's say we have the following DSL: - -```javascript -let searchTree = ContextureClient({ - key: 'root', - type: 'group', - schema: 'collectionNameSchema', - children: [{ - key: 'namequery', - type: 'text', - field: 'name', - operator: 'containsWord', - value: 'text we want to match on the field name', - }] -}) -``` - -We are able to access the `namequery` node by simply calling: - -```javascript -let nameQueryNode = searchTree.getNode(['root', 'namequery']) -``` - -##### Mutate, Add and Remove -Once you have a Context Tree ready, you will trigger searches -automatically by invoking one of the following functions: `add`, -`remove`, `mutate`. More specifically: - -- If you want to run a search after changing some of the properties of - a specific node, you would call `mutate`. -- If you want to add a children to any given `group`, even the root - group, you would call `add`. -- If you want to remove a children to any given `group`, even the root - group, you would call `remove`. - -Let's see some examples: - -Mutate - -Mutate allows us to change properties on nodes and trigger search -afterwards. Here is an example: - -```javascript -await searchTree.mutate(['root', 'namequery'], { - value: 'new value' -}) -``` - -This will change the tree and produce a new set of results. If you're -wondering how to keep track of these changes, the simplest way to do -it is by using our MobX adapter, as shown in our [simple search -box](../../getting-started/simple-search-box.md) tutorial, or in -greater detail in our [Managing State](managing-state/README.md) docs. - -Add - -Having the previous search tree, we can add a children by doing: - -```javascript -await searchTree.add(['root'], { - key: 'results', - type: 'results' -}) -``` - -Remove - -We can remove the `results` node we just added by doing: - -```javascript -await searchTree.add(['root', 'results']) -``` - -Calling `mutate`, `add` or `remove` will trigger events not only for -the node that these functions are targetting, but also for nearby -nodes, depending on the types. - -Up next, we'll dig a bit into what are the client side reactors (the -rules that Contexture Client follows to know what other nodes are -relevant for each update). - -#### Introduction to Reactors -When any node changes, depending on the type, it might be reasonable -to re-trigger a search call for other nodes. We call this process the -selection of `reactors`, where the possible reactors are only three: -`self`, `others` and `all`. - -Reactors should be specified in the Client Types, where each type will -have a specific reactor for each one of it's properties. For example -(taken from our client side [example -types](https://github.com/smartprocure/contexture-client/blob/master/src/exampleTypes.js)): - -```javascript -facet: { - reactors: { - values: 'others', - mode: 'others', - size: 'self', - optionsFilter: 'self', - sort: 'self', - } -} -``` - -The client side type defined above will be effective for any node with -type `facet`, where the properties `values` and `mode` will affect -only all the other nodes (and not itself), and the properties `size`, -`optionsFilter` and `sort` will affect only the specific `facet` node -and no other node. - -The one remaining reactor that isn't covered by that example is the -`all` reactor. The difference between `others` and `all` is that -`others` excludes the node where the change is happening, and `all` -includes all other nodes and the node where the change is happening -(effectively combining `self` and `others`). - - -## Types and Type Components -### DIY Types -The Contexture ecosystem provides a defined list of types that can be -used to perform a wide variety of different searches. Each type we -offer also has a respective component in our `contexture-react` repo. -We've made these components so you can quickstart your search interfaces! - -Even if our types are focused on the different search interfaces we -provide, our API is designed to allow you to build any type you might -need for any other possible use case you might encounter. - -We believe that making a generic framework will allow users to be -creative on their search solutions. Because of that, we will start this -document by explaining how to build your own types. - -#### How to Write a Provider Type -Initializing Contexture Core -requires you to send a bunch of types per provider. A type on any -provider is just a valid string property name on the object that is -sent, accompanied with a corresponding value of a plain JavaScript -Object with one or more of the following properties: - -| Property | Type | Params (Type) | Return Value Type | What it does | -| --- | --- | --- | --- | --- | -| `hasValue` | Function | Node (Object) | Boolean | Allows Contexture to know wether or not to process this search. | -| `filter` | Function | Node (Object) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | -| `result` | Function | Node (Object), Search (Function) | Promise | Allows running a direct search for this type before Comtexture sends the full seaech with the whole tree. | - -Once you have written a type, you can use it by sending it to an -existing Contexture Provider. It -should look more or less like: - -```javascript -let myType = { - hasValue: node => node.requiredProperty, - filter: node => ({ - providerQueryObject: { - value: node.requiredProperty - } - }) -} - -let provider = MyProvider({ - types: { - myType - } -}) -``` - -**Type names are not exclusive across providers**. You can define one -type called `myAwesomeType` in more than one provider and you'll be -able to keep the same required node properties, thus the same -`hasValue`. This allows us to provide the same API for several types, -and re-use code even if we switch the target database of the search. - -Once you have a type defined for one or more providers, you should -write the same type for `contexture-client`. - -#### How to Write a Client Type -Contexture Client already provides a some types based on our -`Example Types` (more on that later.) These type definitions help -the client understand how a specific node affects every other node or -itself. - -To create a custom type, you will need to think on the behaviors you -might need for each one of the following properties: - -| Property Name | Type | Description | -| --- | --- | --- | -| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | -| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our [Introduction to Reactors]() | -| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | - -More details about `contexture-client` types, their properties and -their reserved words can be seen on the README of -[contexture-client](https://github.com/smartprocure/contexture-client#client-types). - -The example types are already included in any instantiation -of Contexture Client's Contexture Tree. However, you can add any type -you need simply by extending the exposed `exampleTypes` with your own. -In the following snippet, we initialize a `ContextureTree` with the -available `exampleTypes`, and our new `myType`: - -```javascript -import * as ContextureClient from 'contexture-client' - -let tree = ContextureClient.ContextTree({ - types: { - ...ContextureClient.exampleTypes, - myType: { - validate: node => node.requiredProperty, - reactors: { - requiredProperty: 'others', - }, - defaults: { - requiredProperty: false - } - } - } -}, { - // Here we will have the underlying tree -}) -``` - -#### How to Write a UI Component for a Type -Writing a user interface for any type can be as simple as writing an -HTML or JSX Element that will render or modify any property of the -any node of an existing Contexture Tree, for example, using our custom -type `myType`, we could write an input field that, onChange, will -write the field's value into the `requiredProperty`. For example: - -```javascript -// This is ES6+ and JSX - -import * as ContextureClient from 'contexture-client' - -let tree = ContextureClient.ContextTree( - { - service: myService, - types: { - ...ContextureClient.exampleTypes, - myType: { - validate: node => node.requiredProperty, - reactors: { - requiredProperty: 'others', - }, - defaults: { - requiredProperty: false - } - } - } - }, { - key: 'root', - join: 'and', - children: [{ - key: 'myNode', - type: 'myType', - }] - } -) - -let node = tree.getNode(['root', 'myNode']) - -let Component = ({ node }) => ( - { - node.requiredProperty = e.target.value - }} - /> -) -``` - -Now that you have a component you can render it and play with it, but -the component won't render by itself. - -#### The Example Types -With the intention of providing practical examples of how to write -types, we decided to share some of the types we use in our production -applications. These types belong to two different database processors: -`contexture-elasticsearch` and `contexture-mongo`. - -**Example Types aren't the rule**. These types are only provided to -serve as a guide to build any other type you might need for your -application. You can also **extend our example types**, simply by -assigning new types as properties on the objects exposed by -`contexture-elasticsearch` and `contexture-mongo`. - -### ElasticSearch Example Types -Contexture is designed to target any database you might need. However, -so far we have only inplemented database providers for the only -databases that we use: ElasticSearch and Mongo. - -Most of our types have relevant components already written to -facilitate building search interfaces. As we progress over each one of -these types, we will show small examples of simple components written -to search with these types. We hope to provide enough information to -allow you to take as little as you need, or as much as you want, and -fulfill you expectations. - -Our ElasticSearch types are the following ones: - -#### Results Type - -The `results` node is a node that if present, idicates to Contextre -that the queries and aggregations should finally run and retrieve all -the filtered values. It allows some customization properties, as -shown below: - -| Property Name | Type | Description | -| --- | --- | --- | -| `page` | Number | Current page of results. | -| `pageSize` | Number | Total results per page. | -| `sortField` | String | Field used to sort the results. | -| `sortDir` | String | Direction of sorting (`asc`, `desc`) | - -#### Bool Type - -The bool type is intended to work as an ElasticSearch terms -aggregation with only one value for a single property. This is useful -for user interfaces with a checkbox to include or exclude a specific -field from a search (or a specific field-value pair). - -Here is the list of all properties this type uses: - -| Property Name | Type | Description | -| --- | --- | --- | -| field | String | Name of the field that we will be using to filter the search search. | -| value | String | Value of the field that will be used to filter the search. | - -Example input: - -```javascript -{ - type: 'bool', - field: 'fieldName', - value: true -} -``` - -Example output: - -```javascript -{ - term: { - fieldName: true - } -} -``` - -You can read more about it in: - -- [Source code of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js). -- [Unit tests of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/bool.js). -- Elastic Search [Term Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html). - -#### Cardinality Type - -The Cardinality type serves to calculate an approximate count of the -distinct available values. This type only uses one property, `field`. -It returns it's values in the `context` of the node, rather than in -the `results` node. - -Example input: -```javascript -{ - type: 'cardinality', - field: 'Organization.Name.untouched' -} -``` - -Example output: -```javascript -{ - aggs: { - cardinality: { - cardinality: { - field: 'Organization.Name.untouched' - } - } - } -} -``` - -You can read more about it in: - -- [Source code of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/cardinality.js). -- [Unit tests of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/cardinality.js). -- Elastic Search [Cardinality Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html). - -#### Date Type - -The Date type is used to specify a range of dates that will be used to -filter the available results. This range of dates can be specified by -a string formatted date (`YYYY-MM-DD`) or by a small set of possible -humanly readable date ranges. Details follow: - -| Property Name | Type | Description | -| --- | --- | --- | -| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | -| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | -| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | -| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | - -Example input 1: -```javascript -{ - type: 'date', - field: 'fieldName', - from: '2016-04-25' -} -``` - -Example output 1: -```javascript -{ - range: { - fieldName: { - gte: '2016-04-25', - format: 'dateOptionalTime' - } - } -} -``` - -Example input 2: -```javascript -{ - type: 'date', - field: 'fieldName', - from: 'thisQuarter', - useDateMath: true, -} -``` - -Example output 2: -```javascript -{ - range: { - fieldName: { - gte: moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD'), - lte: moment.utc(datemath.parse(`${moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD')}||+3M-1d/d`)).format('YYYY-MM-DD'), - format: 'dateOptionalTime' - } - } -} -``` - -You can read more about it in: - -- [Source code of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). -- [Unit tests of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). -- Elastic Search [Range QUery](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html). - -#### Date Histogram Type - -The `dateHistogram` type is very useful for retrieving the number of -results within specific periods of time. The idea here is to end up -building a chart showing how records have changed over time, for -example by it's creation date or some other date property. This type -is able to do this by running a nested stats aggregation inside a -dateHistogram aggregation, while also supporting for tweaking min/max -bounds. - -This type returns it's values in the `context` of the node, rather -than in the `results` node. - -| Property Name | Type | Description | -| --- | --- | --- | -| `key_field` | String | What might be considered a good candidate for the X axis of the chart. | -| `value_field` | String | What might be considered a good candidate for the Y axis of the chart. | -| `interval` | | String | Available expressions for interval: `year` (`1y`), `quarter` (`1q`), `month` (`1M`), `week` (`1w`), `day` (`1d`), `hour` (`1h`), `minute` (`1m`), `second` (`1s`). | -| `boundsRange_min` | Date or String | Lower date limit that will be considered to filter the possible results. | -| `boundsRange_max` | Date or String | Upper date limit that will be considered to filter the possible results. | -| `boundsRange_useDateMath` | Boolean | If set to `true`, the min and max ranges will be valid as strings representative of dates, and will be formatted by NPM's library [`@elastic/datemath`](https://github.com/elastic/datemath-js). | - -Example input: -```javascript -{ - type: 'dateHistogram', - key_field: 'PO.IssuedDate', - value_field: 'LineItem.TotalPrice' -} -``` - -Example output: -```javascript -{ - aggs: { - max_date: { - max: { - field: 'PO.IssuedDate', - }, - }, - min_date: { - min: { - field: 'PO.IssuedDate', - }, - }, - twoLevelAgg: { - date_histogram: { - field: 'PO.IssuedDate', - interval: 'year', - min_doc_count: 0, - }, - aggs: { - twoLevelAgg: { - stats: { - field: 'LineItem.TotalPrice', - }, - }, - }, - }, - }, -} -``` - -You can read more about it in: - -- [Source code of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). -- [Unit tests of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). -- [Two Level Aggregations in ElasticSearch](https://www.elastic.co/blog/intro-to-aggregations-pt-2-sub-aggregations). -- [Date Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html). - -#### Exists Type - -The `exists` type is used to check wether some property exsits or not. -It requires only two fields: `field` and `value`. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The target field we want to check. | -| `value` | Boolean | the value we want to check. Normally true or false. | - -Example input: -```javascript -{ - type: 'exists', - field: 'fieldName', - value: true -} -``` - -Example output: - -```javascript -{ - exists: { - field: 'fieldName' - } -} -``` - -You can read more about it in: - -- [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). -- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). -- Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). - -#### Facet Type - -The `facet` type represents a list of dynamic choices, e.g. a checkbox -list filter. We achieve this by running an ElasticSearch terms -aggregation. We provide a way to limit the number of results that you -will receive with the `size` property, so that large queries can -safely be used. For that same purpose, the property `optionsFilter` is -given, so that search queries can filter the results with a string. - -Facet returns it's values in the `context` of the node, rather than in -the `results` node. - -| Property | Type | Default | Description | -| ---- | ---- | ------- | ----------- | -| `field` | String | None, *required* | The field it's operating on. | -| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | -| `values` | Array (of strings) | `[]` | Already selected values. | -| `fieldMode` | String (`autocomplete`, `word` or `suggest`) | `autocomplete` | Whether to look at the entire field (`autocomplete`), the analyzed words in the field (`word`), or magic suggestions (`suggest`). | -| `size` | Number | 12 | How many options to return. | -| `cardinality` | Number | 5000 | Precision threshold override. | -| `includeZeroes` | Boolean | false | If true, it will include options with 0 matching documents (aka `min_doc_count: 0`) | -| `optionsFilter` | String | '' | Filters the options further, e.g. a find box above a checkbox list | -| `caseSensitive` | Boolean | false | Whether options filter is case sensitive. | -| `sort` | String (`term` or `count`) | `count` | Sort results alphabetically or by count of matching records. | - -Example input 1: -```javascript -{ - type: 'facet', - field: 'fieldName', - values: ['abc', '123'] -} -``` - -Example output 1: -```javascript -{ - terms: { - 'fieldName.untouched': ['abc', '123'] - } -} -``` - -Example input with exclude: -```javascript -{ - type: 'facet', - field: 'fieldName', - mode: 'exclude', - values: ['abc', '123'], -} -``` - -Example output with exclude: -```javascript -{ - bool: { - must_not: { - terms: { - 'fieldName.untouched': ['abc', '123'], - }, - }, - }, -} -``` - -You can read more about it in: - -- [Source code of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). -- [Unit tests of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). -- Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). - -#### Geo Type - -The `geo` type represents a geographic radius search. It needs a -geocodeLocation service passed in to it. We currently assume that you -will be using a google maps geocoder search. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String (required) | The field it's operating on | -| `location` | String (required) | Location to geocode (e.g. an address, businessname, anything the google geocode can take) | -| `radius` | Number (required) | Radius in miles | -| `operator` | String (either `within` or `not within`) | Whether the filter forces inclusion or exclusion (defaults with `within`). | - -Example input: -```javascript -{ - type: 'geo', - field: 'fieldName', - location: 'SmartProcure', - radius: 10, - operator: 'within', -} -``` - -Example output: - -```javascript -{ - geo_distance: { - fieldName: '26.3170479,-80.1131784', - distance: '10mi', - }, -} -``` - -You can read more about it in: - -- [Source code of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/geo.js). -- [Unit tests of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/geo.js). -- [ElasticSearch's Geo Distance Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-query.html). - -#### Number Type - -Number represents a number range with inclusive bounds. This type -provides the ability to determine the best range values based on -percentile interval and range threshold. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `min` | Number | Lower boundary of the filter. | -| `max` | Number | Upper boundary of the filter. | - -Some Notes: -1. An empty value as the upper boundary represents infinity. -2. An empty value as the lower boundary represents negative infinity. -3. Zero has to be respected as a boundary value. -4. If findBestRange is true it will return the best min and max range. - -Example input: -```javascript -{ - type: 'number', - field: 'fieldName', - min: 500, - max: 1000, -} -``` - -Example output: - -```javascript -{ - range: { - fieldName: { - gte: 500, - lte: 1000, - }, - }, -} -``` - -You can read more about it in: - -- [Source code of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). -- [Unit tests of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). - -#### Number Range Histogram Type - -The type `numberRangeHistogram` represents a number range with inclusive bounds. This type -returns feedback in the form of histogram and statistical data. -This type returns it's values in the `context` of the node, rather -than in the `results` node. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `min` | Number | Lower boundary of the filter. | -| `max` | Number | Upper boundary of the filter. | -| `percentileInterval` | Number | Used to group the results based on how many of the records are within each one of the sections given by the interval, from the `min` value, up to the `max` value. | - -Some Notes: -1. An empty value as the upper boundary represents infinity. -2. An empty value as the lower boundary represents negative infinity. -3. Zero has to be respected as a boundary value. - -- [Source code of the type: numberRangeHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/numberRangeHistohgram.js). -- [Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/search-aggregations-bucket-histogram-aggregation.html). - -#### Query Type - -Query represents a raw elasticsearch query_string. It's mostly used to -provide a simple sarch box on the user interface. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | -| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches (accepting values if they contain the given query, even if the casing doesn't match). | - -Example input: -```js -{ - type: 'query', - field: 'fieldName', - query: 'cable', - exact: true, -} -``` - -Example output: -```js -{ - query_string: { - query: 'cable', - default_operator: 'AND', - default_field: 'fieldName.exact', - analyzer: 'exact', - }, -} -``` - -You can read more about it in: - -- [Source code of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). -- [Unit tests of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). - -#### Text Type - -Text implements raw text analysis like starts with, ends with, etc. -These are generally regex queries. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `join` | String | Either `any`, `all` or `none`. | -| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | -| `values` | Array | Array containing all the words that want to be used as inputs. | - -Example input: -```js -{ - type: 'text', - field: 'fieldName', - join: 'any', - operator: 'contains', - values: ['laserjet', 'printer'], -} -``` - -Example output: -```js -{ - query_string: { - default_field: 'fieldName', - default_operator: 'OR', - query: '"laserjet" "printer"', - }, -} -``` - -You can read more about it in: - -- [Source code of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/text.js). -- [Unit tests of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/text.js). - -#### Other ElasticSearch Example Types - -For more informaion about other available example types, please check: - - -### Mongo Example Types -Most of our types have relevant components already written to -facilitate building search interfaces. As we progress over each one of -these types, we will show small examples of simple components written -to search with these types. We hope to provide enough information to -allow you to take as little as you need, or as much as you want, and -fulfill you expectations. - -Our MongoDb types are the following ones: - -#### Results Type - -The `results` node is a node that if present, idicates to Contextre -that the queries and aggregations should finally run and retrieve all -the filtered values. It allows some customization properties, as -shown below: - -| Property Name | Type | Description | -| --- | --- | --- | -| `page` | Number | Current page of results. | -| `pageSize` | Number | Total results per page. | -| `sortField` | String | Field used to sort the results. | -| `sortDir` | String | Direction of sorting (`asc`, `desc`) | - -#### Date Type - -The Date type is used to specify a range of dates that will be used to -filter the available results. This range of dates can be specified by -a string formatted date (`YYYY-MM-DD`) or by a small set of possible -humanly readable date ranges. Details follow: - -| Property Name | Type | Description | -| --- | --- | --- | -| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date (formatted properly for ElasticSearch), or one of the following values: `thisQuarter` (for the current quarter of the year), `lastQuarter` (for the previously completed quarter of the year), `nextQuarter` (for the upcoming quarter). | -| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date (formatted properly for ElasticSearch). | -| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | -| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | - -Example input: -```javascript -{ - type: 'date', - field: 'fieldName', - from: '2016-04-25' -} -``` - -Example output: -```javascript -{ - fieldName: { - $gte: new Date('2016-04-25'), - }, -} -``` - -You can read more about it in: - -- [Source code of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). -- [Unit tests of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). -- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). -- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). - -#### Exists Type - -The `exists` type is used to check wether some property exsits or not. -It requires only two fields: `field` and `value`. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The target field we want to check. | -| `value` | Boolean | the value we want to check. Normally true or false. | - -Example input: -```javascript -{ - type: 'exists', - field: 'fieldName', - value: true -} -``` - -Example output: - -```javascript -{ - $and: [ - { - fieldName: { - $exists: true, - $ne: '', - }, - }, - { - fieldName: { - $ne: null, - }, - }, - ], -} -``` - -You can read more about it in: - -- [Source code of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/exists.js). -- [Unit tests of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/exists.js). -- MongoDb's [$exists](https://docs.mongodb.com/manual/reference/operator/query/exists/). - -#### Facet Type - -The `facet` type represents a list of dynamic choices, e.g. a checkbox -list filter. We achieve this by running an ElasticSearch terms -aggregation. We provide a way to limit the number of results that you -will receive with the `size` property, so that large queries can -safely be used. For that same purpose, the property `optionsFilter` is -given, so that search queries can filter the results with a string. - -Facet returns it's values in the `context` of the node, rather than in -the `results` node. - -| Property | Type | Default | Description | -| ---- | ---- | ------- | ----------- | -| `field` | String | None, *required* | The field it's operating on. | -| `mode` | String (`include` or `exclude`) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | -| `values` | Array (of strings) | `[]` | Already selected values. | -| `size` | Number | 10 | How many options to return. | - -Example input: -```javascript -{ - type: 'facet', - field: 'fieldName', - values: ['abc', '123'] -} -``` - -Example input with exclude: -```javascript -{ - type: 'facet', - field: 'fieldName', - mode: 'exclude', - values: ['abc', '123'], -} -``` - -You can read more about it in: - -- [Source code of the type: facet](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). - -#### Number Type - -Number represents a number range with inclusive bounds. This type -provides the ability to determine the best range values based on -percentile interval and range threshold. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `min` | Number | Lower boundary of the filter. | -| `max` | Number | Upper boundary of the filter. | - -Some Notes: -1. An empty value as the upper boundary represents infinity. -2. An empty value as the lower boundary represents negative infinity. -3. Zero has to be respected as a boundary value. -4. If findBestRange is true it will return the best min and max range. - -Example input: -```javascript -{ - type: 'number', - field: 'fieldName', - min: 500, - max: 1000, -} -``` - -Example output: - -```javascript -{ - fieldName: { - $gte: 500, - $lte: 1000 - } -} -``` - -You can read more about it in: - -- [Source code of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). -- [Unit tests of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). -- MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). -- MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). - -#### Text Type - -Text implements raw text analysis like starts with, ends with, etc. -These are generally regex queries. - -| Property Name | Type | Description | -| --- | --- | --- | -| `field` | String | The field we will be using to filter the results. | -| `join` | String | Either `any`, `all` or `none`. | -| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | -| `values` | Array | Array containing all the words that want to be used as inputs. | - -Example input: -```js -{ - type: 'text', - field: 'fieldName', - join: 'any', - operator: 'contains', - values: ['laserjet', 'printer'], -} -``` - -Example output: -```js -{ - $or: [{ - fieldName: { - $regex: 'laserjet', - $options: 'i' - } - }, { - fieldName: { - $regex: 'printer', - $options: 'i' - } - }] -} -``` - -You can read more about it in: - -- [Source code of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/text.js). -- [Unit tests of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/text.js). - -#### Other MongoDb Example Types - -For more informaion about other available example types, please check: - - -### Available React Components for Types -A huge part of working with advanced search interfaces is identifying -how to portray a specific search filter that you might want to use. -We've talked about very simple components that react to changes on the -tree state, but now we'll see how to make it easier for more complex -nodes to gather the inputs necessary, and also to show the results -correctly. Here we'll see components specifically crafted for some of -our providers' example types. - -**Notes:** -- Keep in mind that the theme on these components is purely -optional. -- Please be aware that when we refer to `Component`, we mean a - Function that returns a valid JSX Element. You can read more here: - [Components and - Props, on the ReactJS docs](https://reactjs.org/docs/components-and-props.html). - -#### Query - - - -The Query component is probably the most commonly needed for search -interfaces. It's an input field that allows you to filter results if -the text matches part of the value of a specfic property on any of the -records. - -Here's how you write a node of type `query` in your _searchTree_: -```javascript -{ - key: 'searchQuery', - type: 'query', - field: 'title', - query: '' -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `query` | String | No | Search query that should be visible in the component. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. - -Here's how you write your component: -```javascript -let Query = require('contexture-react/dist/exampleTypes').Query -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `TextInput` | Component | `input` | Text input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). -- [(ElasticSearch Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). -- [(MongoDb Provider) Source code of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/query.js). -- [(MongoDb Provider) Unit tests of the query type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/query.js). -- [Source code of the Query component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Query.js). - -#### Date - - - -_(Same goes with the right, not adding another screenshot to avoid -consuming more space.)_ - -The Date component helps by allowing users to filter the data to -obtain results within a specific range of dates. It consists of only -two date pickers. Here's how you write a node of type `date` in your -_searchTree_: -```javascript -{ - type: 'date', - field: 'fieldName', - from: '2016-04-25', - to: '2017-05-26' -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `from` | Date or String (`YYYY-MM-DD` format) | Yes | The initial date of our timeframe. | -| `to` | Date or String (`YYYY-MM-DD` format) | No | The final date of our timeframe. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. - -Here's how you write your component: -```javascript -let DateComponent = require('contexture-react/dist/exampleTypes').Date -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `DateInput` | Component | `x => ` | The component that wraps each one of the inputs where the dates end up written by the date-picker. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). -- [(ElasticSearch Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). -- [(MongoDb Provider) Source code of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). -- [(MongoDb Provider) Unit tests of the date type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). -- [Source code of the Date component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Date.js). - -#### DateHistogram - - - -The DateHistogram component is about representing how many records -were found during what periods of time. This component currently -doesn't offer interactive features, but you could use it as -inspiration to write another one that would allow you to dive in these -date ranges to see what records appear for each one of them. - -Here's how you write a node of type `dateHistogram` in your _searchTree_: -```javascript -{ - key: 'releases', - type: 'dateHistogram', - key_field: 'released', - value_field: 'imdbVotes', - interval: '3650d', -} -``` - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. - -Since this component doesn't need any property from the node itself, -but only from the results, here's how you write your component: -```javascript -let DateHistogram = require('contexture-react/dist/exampleTypes').DateHistogram -let formatYear = x => new Date(x).getFullYear() + 1 -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `background` | Function | `(record, max) => '#ccc'` | A function that returns the background color that is used to render each one of the bars in the resulting chart. | -| `height` | Number | `100` | Specifies the max height of the whole chart. | -| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | -| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | -| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). -- [(ElasticSearch Provider) Unit tests of the dateHistogram type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). -- [Source code of the DateHistogram component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/DateHistogram.js). - -#### Facet - - - -The Facet component allows users to filter the results by picking a -specific common option among all the values that the records might -have for a specific field. - -Here's how you write a node of type `facet` in your _searchTree_: -```javascript -{ - key: 'facet', - type: 'facet', - values: ['a'], -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `values` | Array | Yes | Array of selected values. To have a value selected by default, you will need to know in advance which value to put here, otherwise you can leave this with an empty array and let users select the values themselves. | -| `size` | Number | No | Max number of options to display. The default is 10. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. - -Here's how you write your component: -```javascript -let Facet = require('contexture-react/dist/exampleTypes').Facet -// ... -// Later, on your render function, or where you put your components: - -``` - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `hide.facetFilter` | Object (with an optional single property, `facetFilter`) | `{}` | Allows you to hide the text input that helps on searching for the available options. This text input is very valuable when the results are larger than the available visible results. | -| `TextInput` | Component | `input` | Allows you to customize the text input. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). -- [(ElasticSearch Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). -- [(MongoDb Provider) Source code of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). -- [(MongoDb Provider) Unit tests of the facet type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/facet.js). -- [Source code of the Facet component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Facet.js). - -#### Number - - - -The Number component allows users to specify a numeric range to filter -the data based on the available results which values fit within this -range for a specific field. - -Here's how you write a node of type `number` in your _searchTree_: -```javascript -{ - key: 'searchNumber', - type: 'number', - field: 'metaScore', - min: 0, - max: 100, -} -``` - -Here is the list of properties that this component expects to have on the node: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `min` | Number | No | Minimum number for the range. | -| `max` | Number | No | Maximum number for the range. | - -**Note:** The properties present in the search tree that aren't used by the node -might be needed for the Provider's type. - -Here's how you write your component: -```javascript -let Number = require('contexture-react/dist/exampleTypes').Number -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `NumberInput` | Component | `x => ` | Number input component. Useful for any style customization, and for libraries that (for example) wrap Bootstrap and it's classes. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). -- [(ElasticSearch Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). -- [(MongoDb Provider) Source code of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). -- [(MongoDb Provider) Unit tests of the number type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). -- [Source code of the Number component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/Number.js). - -#### ResultCount - - - -The ResultCount component will only show you the number of visible -results compared to the number of total results. It's not an -interactive component. - -Most of your Contexture Trees will have a node with type `results`. -This node posesses information such as the resulting records -themselves, but also which page you're in, how many -elements are per page you will receive, and the total records there -are for this given query, which is what we're looking for for this -type. - -All of these properties are automatically writen by the Contexture -architecture, so we'll be simply passing the tree and the path to the -results type node: - -```javascript -let ResultCount = require('contexture-react/dist/exampleTypes').ResultCount -// ... -// Later, on your render function, or where you put your components: - -``` - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). -- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). -- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). -- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). -- [Source code of the ResultCount component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultCount.js). - -#### ResultPager - - - -The ResultPager component is an interactive component that will show -you which page you're at (in the middle) and what pages are around -your current page, as well as some controllers to move forward and -backwards. - -The style of this component isn't very friendly, but it's very easy to -customize. For more about theme changes, please visit our -[theming docs](../theming/README.md). - -Most of your Contexture Trees will have a node with type `results`. -This node posesses information such as the resulting records -themselves, but also which page you're in, how many -elements are per page you will receive, and the total records there -are for this given query. We use a combination of those values to get -how many pages we can move forward (and backwards), and also to move -around these pages (since our trees react in real time). - -All of these properties are automatically writen by the Contexture -architecture, so we'll be simply passing the tree and the path to the -results type node: - -```javascript -let ResultPager = require('contexture-react/dist/exampleTypes').ResultPager -// ... -// Later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `List` | Component | `div` | The component that wraps the whole list of pages. | -| `Item` | Component | `span` | The component that wraps each of the available page numbers. | -| `Link` | Component | `a` | An element that wraps each one of the pagination controls. | - -To read more, check the following links: - -- [(ElasticSearch Provider) Source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). -- [(ElasticSearch Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). -- [(MongoDb Provider) Source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). -- [(MongoDb Provider) Unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). -- [Source code of the ResultPager component](https://github.com/smartprocure/contexture-react/blob/master/src/exampleTypes/ResultPager.js). - -#### ResultTable - -The ResultTable is a component that will display a table with all the -available results, in which each of the result values will be -displayed as columns. - -the results are automatically writen by the contexture architecture, -so we'll be simply passing the tree and the path to the results type -node: - -```javascript -let resulttable = require('contexture-react/dist/exampletypes').resulttable -// ... -// later, on your render function, or where you put your components: - -``` - -This component can be customized by passing any of the following -properties: - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `Table` | Component | `table` | The component that wraps the table list of results. | - -To read more, check the following links: - -- [(ElasticSearch Provider) source code of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/results.js). -- [(elasticSearch Provider) unit tests of the results type](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/results.js). -- [(MongoDb Provider) source code of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js). -- [(MongoDb Provider) unit tests of the results type](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/results.js). -- [Source code of the ResultTable component](https://github.com/smartprocure/contexture-react/blob/master/src/exampletypes/ResultTable.js). - -## Other Components -### Generally Useful Components -#### ContextureProvider - -This component is a magic tool to make Contexture search interfaces without -having to write the tree separated. With this component, you can pass the tree -directly as a plain object, and any children you pass will receive the `tree` -property directly. - -This component receives: - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `types` | Object | Yes | Your client-side types, such as the default types of the Contexture Client. | -| `service` | Object | Yes | The search function that sends the DSL to the initialized Contexture Core. | -| `nodeKey` | Object | Yes | Key for the root node. Defaults with `root`. | -| `...props` | Any other property (children excluded) | Yes | Any other property that you might want to send to the initialization of the Contexture Client. | - -**Note:** Any of the Contexture React Example Types components automatically -add the nodes to the tree if the referenced node is not present. _This is -an experimental feature_. - -Here's how you write your component: -```javascript -let ContextureProvider = require('contexture-react/dist/ContextureProvider') -let ContextureClient = require('contexture-client') -let types = ContextureClient.exampleTypes -let service = async search => ({ - data: await postData('/sarch', { search }) -}) -// ... -// Later, on your render function, or where you put your components: - - - -``` - -- [Source code of the ContextureProvider component](https://github.com/smartprocure/contexture-react/blob/master/src/ContextureProvider.js). - -#### FilterList - -A component that tries to automatically render the specific type components of -the children of a node. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `node` | Object | Yes | Node of the Contexture Tree where the children that want to be rendered are. | -| `exampleTypes` | Object | No | Object which a key and a component per type. Defaults in the available example types components in contexture-elasticsearch. | -| `fields` | Object | Yes | Object which a key and an object with at least a `label` string property, used to indicate the label of each one of the filters. | - -Here's how you write your component: -```javascript -let FilterList = require('contexture-react/dist/FilterList') -let ContextureClient = require('contexture-client') -let tree = ContextureClient({ - key: 'root', - type: 'group', - schema: 'mySchema', - children: [{ - key: 'query', - type: 'query', - field: 'myFieldName', - query: 'Something' - }] -}) -// ... -// Later, on your render function, or where you put your components: - -``` - -- [Source code of the FilterList component](https://github.com/smartprocure/contexture-react/blob/master/src/FilterList.js). - -### Layout Components -#### Awaiter - -Renders a loading indicator until a Promise is resolved. It will ender -an exception (currently just `Ooops...`) if the Promise fails. if the -Promise passes, the children are rendered. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `promise` | Promise | Yes | Minumum number for the range. | - -Here's how you write your component: -```javascript -let Awaiter = require('contexture-react/dist/layout/Awaiter') -let promiseMaker = () => new Promise((resolve, reject)) /* ... */ resolve()) -// ... -// Later, on your render function, or where you put your components: - -
My Crazy Children
- -``` - -- [Source code of the Awaiter component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/Awaiter.js). - -#### BarChart - -Allows you to build your own bar charts (besides just relying on -the DateHistogram component). - -| Property Name | Type | Default Value | Description | -| --- | --- | --- | --- | -| `valueField` | String | `''` | The field of each record where the value used for the height of each barnis located. | -| `categoryField` | String | `''` | The field of each record where the label of each bar is located. | -| `data` | Array (of Objects with both the `valueField` and the `categoryField`) | `[]` | The data that is going to be used to build the chart. | -| `height` | Number | `100` | Specifies the max height of the whole chart. | -| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | -| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | -| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | - -Here's how you write your component: -```javascript -let BarChart = require('contexture-react/dist/layout/BarChart') -// ... -// Later, on your render function, or where you put your components: - -``` - -- [Source code of the BarChart component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/BarChart.js). - -#### SpacedList - -Wraps every children in a div with a given style. Useful for (as the -name portrays) making a spaced list. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `style` | Object | No | The style that will be applied to each div. Defaults in `{ marginBottom: '25px' }`. | - -Here's how you write your component: -```javascript -let SpacedList = require('contexture-react/dist/layout/SpacedList') -// ... -// Later, on your render function, or where you put your components: - -

Hi

- - -``` - -- [Source code of the SpacedList component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/SpacedList.js). - -#### TextHighlight - -Used to highlight content within a text basef on a pattern. - -| Property Name | Type | Required | Description | -| --- | --- | --- | --- | -| `pattern` | String | No | RegExp pattern used to find matching content. | -| `text` | String | Yes | Text used as the target of the matches. | -| `Wrap` | Component | No | Component used to wrap each one of the -matched. Defaults to `i`. | - -Here's how you write your component: -```javascript -let TextHighlight = require('contexture-react/dist/layout/TextHighlight') -// ... -// Later, on your render function, or where you put your components: - -``` - -- [Source code of the TextHighlight component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/TextHighlight.js). - -## Under the Hood -### Design Principles -- Intentionally stateless. -- Detached from state management tools. -- Can work well with state management tools. -- Modern ES6+ -- Non-strict functional programming. -- Configuration based architecture. -- Focus on a very extensible small core. -- Small DSL. - - Database agnostic. - - Isomorphic Tree State. - - Optimized for database discovery. - - Optimized for advanced search interfaces. - - Aiming to be effective for arbitrarily complex database indexes. - - Simplicity over performance. -- Reaction management. - - Tree walking. - - State flags. - - What updates. - - Pauses. - -### Contexture Core -The core of Contexture is a package of its own. Located at [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture), it offers the very underlying function that is designed to process every one of the search queries. It begins with a simple curried function which, once the providers and the schemas are received, proceeds to process the [Contexture DSL](#TODO) by walking down the given search tree, cleaning up every node of possible inconsistencies, then mutating the tree with the directions given by the provider types and schemas, up until a valid search query is obtained. This query is delivered to the provider `runSearch` method, and the result is finally added back to the tree. - -With this in mind, let's get some specifications. - -#### Default Export -Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. - -The first argument is expected to be a plain JavaScript Object with two keys: - -- `providers`: Should be an object where each key will have a [Contexture Provider](#TODO). -- `schemas`: Should be an object where each key will have a [Contexture Schema](#TODO). - -Calling this function with this object only will return another function, which can be used as an asynchronous search runner. You can also pass in all the arguments as once, but the separation of parameters makes it easier to scope setting up the database providers, the types and the schemas from the search execution. - -Example declaration of a search function by passing the schema & providers object first: - -```javascript -const search = Contexture({ - schemas: { - ...mongoSchemas, - ...elasticSearchSchemas, - }, - providers: { - mongo: require('contexture-mongo')({ /* provider configuration */ }), - elasticsearch: require('contexture-elasticsearch')({ /* provider configuration */}), - }, -}) - -// How you might use it with an express-like API: -// app.use('/search', async req, res) => -// res.send(await search(req.body.search)) -``` - -The other two parameters are the search tree (the Contexture -DSL), and an optional object that will be sent along to the -provider's `runSearch` function as the first parameter, that can -contain any property, but that should at least contain the following -properties: - -| Option | Description | -| ------ | ----------- | -| `debug`| Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info. | -| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example. | - -This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. - -#### Process Algorithm - -For each of these steps, walk the tree in a parent-first DFS traversal, with each function optionally asynchronous by returning a promise. Along the way, intermediate data is added to contexts on an object called `_meta`. For each context, every type/processor combination is pulled on the fly, meaning it will use the correct local `Provider` and `Type` info even if some contexts have different schemas[^MultiSchema] - -- Clean/Prep everything (adding `_meta`, etc) -- Add `materializedPaths` (used later by `relevantFilters`) -- Run `filter` for each item if it `hasValue` -- Add `relevantFilters` for each item (all filters combined in their groups by the `groupCombinator` except for their own filters and any filters related to them in the tree via an `OR` -- Get `result` for each item if it `hasValidContext` and is not `filterOnly` (as determined by the client event architecture), passing a pre curried search function that includes `relevantFilters` so types don't need to worry about it - logging each request on `_meta.requests` -- Combine `took` values for all requests to get an accurate number and pass results to `onResult` as they come in if they are defined -- Unless in `debug` mode, scrub off `_meta` from the response - -#### Providers -All `Provider` must specify the following properties: - -- `groupCombinator` - - A function that takes the group and an array of its filters and returns them combined. -- `runSearch` - - A function that actually runs the search. It takes the current context, schema, filters (as processed by the `relevantFilters` function and combined with `groupCombinators`), and the criteria for the current context's results (eg `aggs` for an es aggregation or `highlight` for es results). This function can conditionally do different things based on what it is passed - like knowing if it should run an aggregation or a scan/scroll. -- `types` - - A object hash of the `Provider`'s type implementations. It can optionally include a type called `default` whose properties will be used if one of its types are missing something (e.g specifying the default behavior of `validContext` to always allow or prevent results from running) - -Additionally, a provider may expose config for it's client (e.g. `hosts` or request `timeout` for elasticsearch). - -#### Types -All `Types` can implement any if the following properties. All are optional: - -- `filter` - - Takes the current context and produces the filter that will apply to other data contexts in the group (except those related via `OR`). Typically JSON but can be a string as in the SQL case. -- `hasValue` - - Takes the current context and returns a truthy value for whether or not it has a value -- `result` - - Takes the current context, a curried version of the provider's `runSearch` with filters and everything pre-applied (so it is really easy to run searches), the current schema, and the current provider for advanced use cases. This can run one or more async calls - as long as it returns a promise for the final result. If you need to do additional filtering logic, you can use `runSearch` on the provider directly instead of the convenient curried version and inspect the `_meta.relevantFilters` property to see which filters would have been auto-applied, allowing you to do literally any kind of search you want - but there hasn't been a need for this yet. -- `validContext` - - Takes the current context and returns a truthy value for whether or not it should get results. - -[^MultiSchema]: This completely solves and obviates the need for the `MultiIndexGroupProcessor` on the client and handles it in much more elegant way (and in a single service call, instead of `n` services calls). A caveat is that it does not currently handle schemas from different providers (because filters are generated based on their context's local schema), so you can't currently mix a elasticsearch schema with a mongo schema (because it could try to call mongo with elastic search filters for example). - -#### Schemas -Schemas are named by convention based on their filename and should be in `camelCase`. A schema must have one or more provider specific set of configuration properties. - -### Contexture Client -#### Process Algorithm - -- An action method is called at the top level which: - - Interally mutates the tree - - Makes one or more calls to `dispatch` with relevant event data -- For each dispatched event: - - Validate the entire tree (an async operation) - - Bubble up the tree from the affected node up to the root, and for each node in the path: - - Determine affected nodes by calling the reactor for the current event type - - Mark each affected node for update, or if it is currently paused, mark that it missed updates - - Trigger an update (which is debounced so it does not run right away) -- When the debounce elapses, an update is triggered: - - Check if the update should be blocked - - There may be no affected nodes - - Some nodes might have erros on validation - - Prepare for update - on each node that's markedForUpdate: - - Set the lastUpdateTime to now (to enable dropping stale results later in this process) - - Set `updating` to true - - Serialize the search, omitting all temporary state except lastUpdateTime (which the sever will effectively echo back) and deleting nodes that are filter only with no value - - Execute an actual contexture search - - For each node in the response: - - If the path isn't found in the current tree, ignore it - - If the response is empty, ignore it - - If the response has a lastUpdateTime earlier than the node in the current tree, ignore it (because it's stale) - - If not ignoring the update, mutate the node with the result and set `updating` to false -- After all of this, the promise for the action/dispatch resolves (so you can await the entire process) - -#### Flat Trees - -The client maintains a flat tree in addition to the actual tree, which is an object mapped using `flattenTree` from `futil-js`. -The keys are the array paths encoded as a string, currently using a slashEncoder. -This allows path lookups to perform in constant time at `O(1)`, drastically speeds up some of the internal tree operations. -The paths are also stamped on individual nodes for convenience as performing an action on a node requires knowing its path. - -#### Initialization -On instantiation, the client creates a flat tree representation of the tree and stamps the paths on the nodes. - -## Examples -- [contexture-site](https://contexture.site) - Showcase of Contexture searches adapting to diverse datasets such as: car crashes, restaurant locations, COVID-19, SAT scores, school grants, bank failures. -- [contexture-imdb](https://github.com/smartprocure/contexture-imdb) - An example usage of Contexture to present a search interface for an ElasticSearch index of movie records based on IMDB. -- [GovSpend](https://app.govspend.com/) - A Contexture web-based tool that makes it easy for government agencies to find a product's best price, identify and validate vendors, request quotes and connect with peers. - - -## Contributing Guide -See the [SmartProcure Contributor Coding Guidelines](https://github.com/smartprocure/contributor-guide). - -## License -Contexture is licensed under [the MIT License](https://github.com/smartprocure/contexture/blob/master/LICENSE). diff --git a/SUMMARY.md b/SUMMARY.md new file mode 100644 index 0000000..c2547c5 --- /dev/null +++ b/SUMMARY.md @@ -0,0 +1,44 @@ +# Table of contents + +* [Contexture Documentation](README.md) + +## About Contexture + +* [What Is Contexture](about-contexture/what-is-contexture.md) +* [Glossary of Terms](about-contexture/glossary-of-terms.md) +* [Map of Repos](about-contexture/map-of-repos.md) + +## Getting Started + +* [Project Setup](getting-started/project-setup.md) +* [Your First Contexture Script](getting-started/your-first-contexture-script.md) +* [Connecting to Elasticsearch & MongoDB](getting-started/connecting-to-elasticsearch-and-mongodb.md) + +## Querying + +* [Untitled](querying/untitled.md) + +## Types and Type Components + +* [Untitled](types-and-type-components/untitled.md) +* [README](untitled/README.md) + * [building-your-own-provider](untitled/building-your-own-provider.md) + * [connecting-other-databases](untitled/connecting-other-databases.md) + * [contexture-client](untitled/contexture-client.md) + * [contexture-core](untitled/contexture-core.md) + * [contexture-dsl](untitled/contexture-dsl.md) + * [contexture-elasticsearch](untitled/contexture-elasticsearch.md) + * [contexture-mongo](untitled/contexture-mongo.md) + * [contexture-react](untitled/contexture-react.md) + * [design-principles](untitled/design-principles.md) + * [discovering-the-database](untitled/discovering-the-database.md) + * [diy-types](untitled/diy-types.md) + * [elasticsearch-example-types](untitled/elasticsearch-example-types.md) + * [general](untitled/general.md) + * [imdb-example](untitled/imdb-example.md) + * [Layout Components](untitled/layout-components.md) + * [mongo-example-types](untitled/mongo-example-types.md) + * [reactors](untitled/reactors.md) + * [simple-search-box](untitled/simple-search-box.md) + * [your-first-filter](untitled/your-first-filter.md) + diff --git a/about-contexture/glossary-of-terms.md b/about-contexture/glossary-of-terms.md new file mode 100644 index 0000000..108c9e6 --- /dev/null +++ b/about-contexture/glossary-of-terms.md @@ -0,0 +1,53 @@ +# Glossary of Terms + +## Contexture + +> NOUN 1. The fact or manner of being woven or linked together to form a connected whole. +> 1.1. A mass of things interwoven together; a fabric. +> 1.2. The putting together of words and sentences in connected composition; the construct of a text. +> 1.3. A connected literary structure; a continuous text. + +[Source](https://en.oxforddictionaries.com/definition/us/contexture). + +## Domain-Specific Language \(DSL\) + +> A domain-specific language \(DSL\) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language \(GPL\), which is broadly applicable across domains. + +[Source](https://en.wikipedia.org/wiki/Domain-specific_language). + +## Search Query + +According to Wikipedia, `Search Query`, in the context of the `web`, refers to: + +> ...a query that a user enters into a web search engine to satisfy his or her information needs. Web search queries are distinctive in that they are often plain text or hypertext with optional search-directives \(such as "and"/"or" with "-" to exclude\). They vary greatly from standard query languages, which are governed by strict syntax rules as command languages with keyword or positional parameters. + +[Source](https://en.wikipedia.org/wiki/Web_search_query). + +## Faceted Search + +> Faceted search, also known as faceted navigation or faceted browsing, is a technique used by eCommerce brands to help users analyze, organize, and filter large sets of product inventory based on filters such as size, color, price, and brand. + +[Source](https://www.dynamicyield.com/glossary/faceted-search/). + +> Faceted search, also called faceted navigation or faceted browsing, is a technique for accessing information organized according to a faceted classification system, allowing users to explore a collection of information by applying multiple filters. A faceted classification system classifies each information element along multiple explicit dimensions, called facets, enabling the classifications to be accessed and ordered in multiple ways rather than in a single, pre-determined, taxonomic order. + +[Source](https://en.wikipedia.org/wiki/Faceted_search). + +## Search Filter + +> An extension of faceted search, a search filter is a specific product attribute a visitor can use to refine the search results of a particular category listing, e.g. by size, color, price, or brand. Multiple filters may be applied to take a broad range of products and refine them into a more narrow selection, allowing the end user to retrieve the most relevant search results based on the criteria they’ve selected. + +[Source](https://www.dynamicyield.com/glossary/search-filter/). + +## Configuration Based Development + +> The difference between configuration-driven development and model-driven development is that the former is not restricted to the model of the code such as classes, fields, and relationships. Configuration-driven development \(CCD\) encompasses anything that can be configured within your application. For example, if your architecture dictates that particular business rules must be applied consistently across your application, you can use configuration files to configure and apply those rules. + +[Source](https://www.ibm.com/developerworks/library/wa-configdev/index.html). + +## Functional Logic Programming + +> Functional logic programming is the combination, in a single programming language, of the paradigms of functional programming \(including higher-order programming\) and logic programming \(nondeterministic programming, unification\). This style of programming is embodied by various programming languages, including Curry and Mercury. + +[Source](https://en.wikipedia.org/wiki/Functional_logic_programming). + diff --git a/about-contexture/map-of-repos.md b/about-contexture/map-of-repos.md new file mode 100644 index 0000000..057e155 --- /dev/null +++ b/about-contexture/map-of-repos.md @@ -0,0 +1,40 @@ +# Map of Repos + +The Contexture framework comes to life through a list of repositories that individually specialize in some needed layer for our architecture. We will be using most \(if not all\) of these projects in our upcoming pages. Let's explore the repos we have so far: + +## Contexture Core + +[github.com/smartprocure/contexture](https://github.com/smartprocure/contexture) is where our main DSL processor lives. It is a very small layer that ties everything together. This one receives the information about the different search representations, about the databases involved, and the DSL, then outputs the search results respective to each one of the queries described in a copy of the received DSL. + +You can read more about the core here: + +* [In the repository](https://github.com/smartprocure/contexture). + +## Contexture Providers + +The Contexture Providers are the interfacing layer that ties the Contexture DSL to the targeted databases. So far, we have only two open source providers: + +* [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch), + + and + +* [contexture-mongo](https://github.com/smartprocure/contexture-mongo). + +If you are planning to use either ElasticSearch or MongoDB for your project, the best way to get started is to use those repositories. However, if you need to use Contexture with another database, you will need to implement the provider yourself. We're looking for code contributors, so please don't feel limited to the current available tools. Help us grow together! + +## Contexture Client + +The Contexture Client is responsible for triggering behaviors on the search interfaces by knowing what causes changes in one or more elements of the search tree. It is the key piece of technology that allows our search interfaces to work in real time. + +You can read more about the Contexture Client here: + +* [In the repository](https://github.com/smartprocure/contexture-client). + +## Contexture React + +The Contexture React repository holds a list of components that facilitate building search interfaces. They are mainly graphical representations of the types that exist on our Contexture Providers. + +You can read more about the Contexture React: + +* [In the repository](https://github.com/smartprocure/contexture-client). + diff --git a/about-contexture/what-is-contexture.md b/about-contexture/what-is-contexture.md new file mode 100644 index 0000000..dfb5838 --- /dev/null +++ b/about-contexture/what-is-contexture.md @@ -0,0 +1,14 @@ +# What Is Contexture + +People of the Internet, here we officialy introduce you to `contexture`, our framework for building search interfaces. + +This framework is carefully designed to be a generic solution for a universe of unlimited possible search interfaces. We've started with a minimal set of repositories that are representative of tools that empower our business, but are intended to be merely examples. If anything, our approaches are only use cases, for the potential of this tool is ultimately yours to take. + +A quick search over the Internet would reveal that the word `contexture` means: `the fact or manner of being woven or linked together to form a connected whole` and also `the putting together of words and sentences in connected composition; the construction of a text`. + +Picking `contexture` as the name for this project means that we are trying to expose not only our ultimate intentions, but also more or less how the system is built. The way our projects work is by a DSL that is used to gather different intended search inputs, each one representing some useful abstraction of a search filter \(like an input where you can write a word to be searched, or another where you can filter the search results by one or more options\), then using the values to process a DSL that will end up retrieving values from one or more different databases, then returning these values on the respective sections of the DSL, so that each result can update each one of the components of the user interface. A more detailed description is visible in the following diagram. + +The canonical example of a Contexture Node is faceted search, where you have a checkbox list that is both a filter \(in the sense that it restricts results based on the checked values\) and an aggregation \(which shows the top n values that can be checked\). Contexture allows them to be nested in advanced searches with boolean joins like `and`/`or`/`not`. + +This thought process will become more clear as we progress through the docs. Hopefully, some pages later it will be easy to grasp how we provide a new perspective on building search interfaces, and perhaps even how you can use it to power up your business, just like we have been doing for almost a decade. + diff --git a/getting-started/connecting-to-elasticsearch-and-mongodb.md b/getting-started/connecting-to-elasticsearch-and-mongodb.md new file mode 100644 index 0000000..1244bf0 --- /dev/null +++ b/getting-started/connecting-to-elasticsearch-and-mongodb.md @@ -0,0 +1,238 @@ +# Connecting to Elasticsearch & MongoDB + +Our primary use case for advanced search interfaces is to query data that we store on ElasticSearch and MongoDB. Because of that, we provide two contexture libraries for those databases. In this page we'll examine how to use these repositories to connect to existing Mongo databases and ElasticSearch indexes. + +## Connecting to ElasticSearch + +The followng code example shows how to connect to ElasticSearch: + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-elasticsearch') +let types = require('contexture-elasticsearch/types') +let elasticsearch = require('elasticsearch') +let AgentKeepAlive = require('agentkeepalive') + +let elasticClient = null +let getClient = () => { + if (elasticClient) return elasticClient + elasticClient = elasticsearch.Client({ + minSockets: 1, + maxSockets: 20, + keepAlive: true, + createNodeAgent: (connection, config) => + new AgentKeepAlive(connection.makeAgentConfig(config)) + }) + return elasticClient +} + +let schemas = { + yourCustomSchemaName: { + elasticsearch: { + index: 'SomeIndex', + type: 'SomeType' + } + } +} + +let search = Contexture({ + schemas, + providers: { + elasticsearch: provider({ + getClient, + request: { + headers: { + 'custom-header-app-name': 'my-app-sent-this' + } + }, + types: types() + }) + } +}) +``` + +The code above will provide a working search function `search` that will transform any given query into a working ElasticSearch query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. + +1. The Dependencies + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-elasticsearch') +let types = require('contexture-elasticsearch/types') +let elasticsearch = require('elasticsearch') +let AgentKeepAlive = require('agentkeepalive') +``` + +The first six lines are about requiring dependencies, the first dependency being just `contexture`. The second one is our database provider, `contexture-elasticsearch`. Then, we require the types; even though you will probably have to write your own types, we have some pre-defined types available at `contexture-elasticsearch/types`. Lastly, we require `elasticsearch` and `agentkeepalive`, so we can actually connect to ElasticSearch and keep it connected even if the address becomes unreachable for a while. + +Please feel free to dig around these topics by following these links: + +* [ElasticSearch.js](https://github.com/elastic/elasticsearch-js). +* [AgentKeepAlive](https://github.com/node-modules/agentkeepalive). +* The ElasticSearch Client + +```javascript +let elasticClient = null +let getClient = () => { + if (elasticClient) return elasticClient + elasticClient = elasticsearch.Client({ + minSockets: 1, + maxSockets: 20, + keepAlive: true, + createNodeAgent: (connection, config) => + new AgentKeepAlive(connection.makeAgentConfig(config)) + }) + return elasticClient +} +``` + +Now, we define a utility function to connect to ElasticSearch just once. The idea here is to be able to re-use the same connection instead of using new clients every time a search is executed. We do this by declaring a function that will return `elasticClient` if it has a truthy value, or otherwise instantiate a new elasticsearch client with the given configuration. The configuration we are sending is merely an example and shouldn't be used without understanding what is being expected by the elasticsearch library. More on the following links: + +* [ElasticSearch.js docs on configuration](https://github.com/elastic/elasticsearch-js/blob/master/docs/configuration.asciidoc). +* The Schemas + +```javascript +let schemas = { + elasticsearch: { + index: 'SomeIndex', + type: 'SomeType' + } +} +``` + +As shown above, the next thing we do is to define the schemas. This is an object which properties are the names of each one of the schemas that will be available for the contexture DSL. The schemas for the ElasticSearch provider can specify any or all of the following properties: + +| Option | Type | Description | Required | +| :--- | :--- | :--- | :--- | +| `index` | `string` | Which ES index to use when querying | x | +| `type` | `string` | Which ES type to use when querying | | +| `summaryView` | `function` | Used by `results` to return a summary view instead of the whole document, \(e.g. for indexes with many fields\). Defaults to returning the `hit` property. | | +| `highlight` | `object` | Used by `results` to determine what fields to highlight, and whether or not they are `inline` \(copied over inline on to the source\) or `additional` \(in a list of additional fields that matched\) | | +| `forceExclude` | `array` | Used by `results` to extend the exclude fields provided on the search tree. The extension happens only if the results node has a `forceExclude` flag set to true. | | + +You can read more about these here: + +* [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). +* Our Search Function + +```javascript +let search = Contexture({ + schemas, + providers: { + elasticsearch: provider({ + getClient, + request: { + headers: { + 'custom-header-app-name': 'my-app-sent-this' + } + }, + types: types() + }) + } +}) +``` + +Once we have the schemas set, we can create our `search` function. For this purpose, we will be calling `Contexture` with just one object. This object will have the `schemas`, and the `providers`. The providers will host just one key/value, the one specific for `elasticsearch`. The provider, which we required at the beginning of the script, needs to be called with the `getClient` function we just created and the `types()` that we got from the `contexture-elasticsearch/types` repository. We also show that you can customize the request headers by providing an object that includes the headers keys and values. This `search` function is ready to receive search trees and write the results back! + +This example and many other important details about `contexture-elasticsearch` are accessible in the following links: + +* [contexture-elasticsearch repository](https://github.com/smartprocure/contexture-elasticsearch). + +## Connecting to MongoDB + +The followng code example shows how to connect to MongoDB: + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient + +let schemas = { + yourCustomSchemaName: { + mongo: { + collection: 'SomeCollection' + } + } +} + +let search = null + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +The code above will provide a working search function `search` that will transform any given query into a working MongoDB query, which will be sent to the database to retrieve the data. Let's examine this code in greater detail. + +1. The Dependencies + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient +``` + +The first lines are about requiring dependencies, the first dependency being just `contexture`. The second one is our database provider, `contexture-mongo`. Then, we require the types; even though you will probably have to write your own types, we have some pre-defined types available at `contexture-mongo/types`. Lastly, we require `mongodb` to get the much needed `MongoClient`, so we can actually connect to the database. + +Please feel free to dig around these topics by following these links: + +* [MongoDB's NodeJS Package's API](http://mongodb.github.io/node-mongodb-native/3.0/api). +* The Schemas + +```javascript +let schemas = { + yourCustomSchemaName: { + mongo: { + collection: 'SomeCollection' + } + } +} +``` + +As shown above, the next thing we do is to define the schemas. This is an object which properties are the names of each one of the schemas that will be available for the contexture DSL. The schemas for the MongoDB provider can specify any or all of the following properties: + +| Option | Type | Description | Required | +| :--- | :--- | :--- | :--- | +| `collection` | `string` | The MongoDB collection that will be used to run the queries | x | + +You can read more about these here: + +* [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). +* The MongoDB Client & Our Search Function + +```javascript +let search = null + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +Next we will need to connect to MongoDB. In the previous code, we provide an example in which we connect to `mongodb://localhost:27017`, and using a callback to the `connect` method, we are able to obtain the database client that we need. Once we have this client, we can actually create our `search` function. For this purpose, we will be calling `Contexture` with just one object. This object will have the `schemas`, and the `providers`. The providers will host just one key/value, the one specific for `mongo`. The provider, which we required at the beginning of the script, needs to be called with a `getClient` function that will just return the database client, and the `types()` that we got from the `contexture-mongo/types` repository. With these steps completed, we end up with a function that is ready to receive search trees and write the results back! + +You can read more about these topics in the following links: + +* [Connecting to MongoDB using the native MongoDB driver for NodeJS](http://mongodb.github.io/node-mongodb-native/api-articles/nodekoarticle1.html#getting-that-connection-to-the-database). + +This example and many other important details about `contexture-mongo` are accessible in the following links: + +* [contexture-mongo repository](https://github.com/smartprocure/contexture-mongo). + diff --git a/getting-started/project-setup.md b/getting-started/project-setup.md new file mode 100644 index 0000000..35add78 --- /dev/null +++ b/getting-started/project-setup.md @@ -0,0 +1,58 @@ +# Project Setup + +The whole Contexture Framework is available through NPM. It doesn't have any extra dependency besides Node 9 and some NPM repositories. + +**Note:** as you progress through our documentation, you'll discover that in some cases, you will need only one or two of these repositories. This page just provides the fastest way to start that we couldcome up with. + +## Installing Node 9 and NPM + +NodeJS 9 and NPM can be installed through [the list of previous releases of NodeJS](https://nodejs.org/en/download/releases/). You might get it working with Node 10 \(if so, let us know\). We haven't fully upgraded to Node 10 yet, so until then, we encourage you to at least have a working version of Node 9 in hand. + +An easy way to move from one version to another is with [nvm](https://github.com/creationix/nvm). Here's a command you can run to install nvm: + +```text +curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash +``` + +Please follow [nvm's README](https://github.com/creationix/nvm/blob/master/README.md) for more information. + +## Installing Contexture + +Once you have NodeJS and NPM installed, you'll need either a new folder for your new project with Contexture, or to go to an existing project, then run: + +```text +npm install --save contexture +``` + +## Installing Contexture Client + +To install the `contexture-client` you can run the following command in your project's root folder: + +```text +npm install --save contexture-client +``` + +## Installing Contexture ElasticSearch + +To install the `contexture-elasticsearch` you can also run the following command in your project's root folder: + +```text +npm install --save contexture-elasticsearch +``` + +## Installing Contexture Mongo + +To install the `contexture-mongo` you can also run the following command in your project's root folder: + +```text +npm install --save contexture-mongo +``` + +## Installing Contexture React + +To install the `contexture-react` you can also run the following command in your project's root folder: + +```text +npm install --save contexture-react +``` + diff --git a/getting-started/your-first-contexture-script.md b/getting-started/your-first-contexture-script.md new file mode 100644 index 0000000..9e3beb5 --- /dev/null +++ b/getting-started/your-first-contexture-script.md @@ -0,0 +1,144 @@ +# Your First Contexture Script + +With everything installed, let's see a simple script that runs a simple one-time search: + +```javascript +let contexture = require('contexture') + +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} + +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} + +let result = await contexture({ + schemas, + providers: { + mongo: require('contexture-mongo')({ + types: require('contexture-mongo/types')(), + getClient: () => ({ + // Fake mongo client. + // For this example we only care about + // collection().aggregate([...]).toArray() being a promise. + collection: name => ({ + aggregate: aggregations => ({ + toArray: async () => ['Unrealistic result example'] + }) + }) + }) + }) + }, +}, searchTree) + +console.log(result.children[1].context.response.results) + +// Explore it yourself! +result +``` + +You can also try this same code in Runkit here: [https://runkit.com/sadasant/your-first-contexture-script](https://runkit.com/sadasant/your-first-contexture-script) + +## What it does + +1. Requiring Contexture + +```javascript +let contexture = require('contexture') +``` + +We start by requiring `contexture`. There's nothing much else to see here. + +![Random but cute gif taken out of Google](https://i.chzbgr.com/full/6410885376/hC6033E5D/) + +1. Writing the Schemas + +```javascript +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} +``` + +Immediatly afterwards, we define our schemas. For this specific example, we are going to emulate doing a search on a single collection of a Mongo database. We define `collectionName` as the name of this arbitrary collection. The `schemas` object ends up containing a single schema with a key being `collectionNameSchema`, which is going to be supported by a single provider `mongo`, from which we'll be looking for the `collectionName` collection. We'll learn more about the schemas later on, in Querying → Schemas. + +1. Writing the Contexture DSL + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} +``` + +Our next step is to define a simple search query using Contexture DSL. This search query will have a mandatory root group, which among other features indicates which schema we will be running this query on \(`collectionNameSchema`\). This node will have two children. The first children is going to be our search query. This query will be a `text` type query. We're indicating that we want to run a plain text search that will try to match any record which `name` contains the word `value`. The next node has the `results` type. This is where the results will be written once the search runs. + +1. Getting Results + +```javascript +let result = await contexture({ + schemas, + providers: { + mongo: require('contexture-mongo')({ + types: require('contexture-mongo/types')(), + getClient: () => ({ + // Fake mongo client. + // For this example we only care about + // collection().aggregate([...]).toArray() being a promise. + collection: name => ({ + aggregate: aggregations => ({ + toArray: async () => ['Unrealistic result example'] + }) + }) + }) + }) + }, +}, searchTree) +``` + +The next thing we do is to actually run a one-time search. In this case, we `await` for `contexture`, passing along some very important parameters. First, we pass the schemas we previously defined. Then, we pass a providers object, which has the provider for `mongo`. This `mongo` key will be matched against the `mongo` key that we defined in the schemas, so you could change this property name to something entirely different. When we send the mongo provider, we assign the result of the initialization of `contexture-mongo`, where we send the `contxture-mongo/types` \(which **needs to be called as a function once required**\) and a `getClient` function. In a real schenario, you would just send the object that results of `require('mongodb')`. However, to provide an exceutable example in the browser, we've made a very small Mock of MongoDB where we will return a same object for any collection call, which will only allow fake aggregations, which will have a promise `toArray` function that will return our `Unrealistic result example`. + +Keep in mind that since `contexture` returns a promise, you can change `await contexture({ /* .. */ })` to `contexture({ /* .. */ }).then()`, but you'll need to move the code that we have after the await call into the function that is passed on the `.then()` call. + +1. Exploring the Results + +```javascript +console.log(result.children[1].context.response.results) + +// Explore it yourself! +result +``` + +Finally, we can use our search result! The output of the search will be added to a copy of the original search query. The location of this result will be in the children of the `root` node that has the `results` type, within a `context` object, within a `response` object, on a `results` property. This whole object is going to be exposed in the _runkit_ example for you to play with it. For more information, you can dive in here: + diff --git a/querying/untitled.md b/querying/untitled.md new file mode 100644 index 0000000..5094080 --- /dev/null +++ b/querying/untitled.md @@ -0,0 +1,2 @@ +# Untitled + diff --git a/types-and-type-components/untitled.md b/types-and-type-components/untitled.md new file mode 100644 index 0000000..5094080 --- /dev/null +++ b/types-and-type-components/untitled.md @@ -0,0 +1,2 @@ +# Untitled + diff --git a/untitled/README.md b/untitled/README.md new file mode 100644 index 0000000..c3c4b0a --- /dev/null +++ b/untitled/README.md @@ -0,0 +1,72 @@ +# README + +\# Contexture Documentation + +This repo is designed to host the documentation of Contexture as a whole. + +## Table of Contents + + + +* [Getting Started]() + * [Connecting to Other Databases](connecting-other-databases.md) + * [Simple Search Box](simple-search-box.md) + * [Your First Filter](your-first-filter.md) + * [Discovering the Database](discovering-the-database.md) + * [IMDB Index](imdb-example.md) +* [Querying]() + * [Contexture DSL](contexture-dsl.md) + * [Interactive Queries]() + * [Contexture Client](contexture-client.md) + * [Initializing the Contexture Client](contexture-client.md#initializing-the-contexture-client) + * [Context Tree](contexture-client.md#context-tree) + * [Tree and getNode](contexture-client.md#tree-and-getnode) + * [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) + * [Introduction to Reactors](reactors.md) +* [Types and Type Components]() + * [DIY Types](diy-types.md) + * [How to Write a Provider Type](diy-types.md#how-to-wite-a-provider-type) + * [How to Write a Client Type](diy-types.md#how-to-wite-a-client-type) + * [How to Write a UI Component for a Type](diy-types.md#how-to-write-a-ui-component-for-a-type) + * [The Example Types](diy-types.md#the-example-types) + * [ElasticSearch Example Types](elasticsearch-example-types.md) + * [Mongo Example Types](mongo-example-types.md) + * [Available React Components for Types]() +* [Other Components]() + * [Generally Useful Components](general.md) + * [Layout Components](layout-components.md) +* [Managing State]() + * [DIY State Management](docs/diy.md) + * [MobX](docs/mobx.md) + * [Redux \(Coming Soon\)](docs/redux.md) +* [Theming]() +* [Recommendations]() + * [Architecture](docs/architecture.md) + * [Client vs Server](docs/architecture.md#client-vs-server) + * [Type definitions](docs/architecture.md#type-definitions) + * [Scaling](docs/architecture.md#scaling) + * [Server Side Searches](docs/server-side-searches.md) + * [Searching On an Endpoint](docs/server-side-searches.md#searching-on-an-endpoint) + * [Caching](docs/server-side-searches.md#caching) + * [Client Side Searches](docs/client-side-searches.md) + * [Click to Search](docs/client-side-searches.md#click-to-search) + * [Real Time Searches](docs/client-side-searches.md#real-time-searches) + * [Cascading](docs/client-side-searches.md#cascading) +* [Under the Hood]() + * [Design Principles](design-principles.md) + * [Contexture Core](contexture-core.md) + * [Default Export](contexture-core.md#default-export) + * [The Algorithm](contexture-core.md#the-algorithm) + * [Utility Functions](contexture-core.md#utility-functions) + * [Contexture Providers]() + * [Contexture ElasticSearch]() + * [Contexture Mongo]() + * [Building Your Own Provider](building-your-own-provider.md) + * [Contexture Client](contexture-client.md) + * [State Properties](contexture-client.md#state-properties) + * [Lenses](contexture-client.md#lenses) + * [Reactors in Detail](contexture-client.md#reactors-in-detail) + * [Serialization](contexture-client.md#serialization) + * [Traversals](contexture-client.md#traversals) + * [Contexture React](contexture-react.md) + diff --git a/untitled/building-your-own-provider.md b/untitled/building-your-own-provider.md new file mode 100644 index 0000000..52b2b8c --- /dev/null +++ b/untitled/building-your-own-provider.md @@ -0,0 +1,69 @@ +# building-your-own-provider + +\# Building Your Own Provider + +Writing a new provider consists of writing a function \(or class or any other approach you might like\), that will receive a configuration object with at least the following properties: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `types` | `Object` | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo). | +| `getClient` | `Function` | A function that returns the main database client that you will be using. | + +You might also add any new property if your provider needs it for processing the searches. + +The main provider function is expected to return an Object with: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `types` | `Object` | See the example-types in [contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) and [contexture-mongo](https://github.com/smartprocure/contexture-mongo). | +| `groupCombinator(group, filters)` | `Function` | A function that returns a valid query for grouping individual aggregations. More on that below. | +| `runSearch(options = {}, context, schema, filters, aggs)` | `Function -> Promise` | A function that will be responsible for running each one of the individual searches that the types in the Contexture DSL might indicate. More on that below. | + +## Group Combinator Function + +The `groupCombinator` function is used to transform an incoming query into the underlying database's group combinator. It receives a `group` Object that contains a property `join`, which indicates which boolean operation wants to be used. The `join` values we use are `and`, `or` and `not`, but you can specify any `join` value for your provider, as long as you make sure the resulting query is valid. The result should be a native database query structure with the given `filters` within a boolean operator. For example, in ElasticSearch, this means: + +| Boolean Operation | ElasticSearch Combinator | +| :--- | :--- | +| `and` | `{ bool: { must: filters } }` | +| `or` | `{ bool: { should: filters, minimum_should_match: 1 } }` | +| `not` | `{ bool: { must_not: filters } }` | + +However, in MongoDB, this is how it works: + +| Boolean Operation | MongoDB Combinator | +| :--- | :--- | +| `and` | `{ $and: filters }` | +| `or` | `{ $or: filters }` | +| `not` | `{ $nor: filters }` | + +## Run Search Function + +The `runSearch` function will run each search that is indicated by the Contexture DSL. It receives several parameters: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `options` | `Object` | Object that is sent as the last parameter to the main `Contexture` call. | +| `context` | `Object` | The current node that is triggering the search. | +| `schema` | `Object` | The schema that is related to this node or root. Usually assigned at the root of the search tree. | +| `filters` | `Array` | The filters that have resulted from parsing this node and it's children. | +| `aggs` | `Array` | Any other aggregation that might have been received from previous processing operations of the Contexture DSL. | + +In this function, you should call to the `getClient` function that the provider originally received, then you should accomodate the incoming `filters` and `aggs` into the final query that will be sent to the client's aggregation method. Since you'll have the schema, you'll be able to get the information you need for your specific database with `schema.myDatabase.myProperty`. + +The final search query should be an Object and should be pushed into the `context._meta.requests` array, for debugging purposes. Once you have the result, you should also add it to the final query object, so that it can be accessible by reading the context path, then `_meta.requests[index].response`, where `index` is the position where the final query lives. For example: + +```javascript +let request = { + /* My Database DSL Object */ +} + +context._meta.requests.push(request) + +let result = Promise.resolve(databaseClient(request)) + +return result.tap(results => { + request.response = results +}) +``` + diff --git a/untitled/connecting-other-databases.md b/untitled/connecting-other-databases.md new file mode 100644 index 0000000..7194332 --- /dev/null +++ b/untitled/connecting-other-databases.md @@ -0,0 +1,6 @@ +# connecting-other-databases + +\#\# Connecting to Other Databases + +Contexture depends on it's providers to be able to know how to translate from the Contexture DSL to the specific DSL that each database needs. Because of this, to connect to other databases you will need to create a new Provider. + diff --git a/untitled/contexture-client.md b/untitled/contexture-client.md new file mode 100644 index 0000000..13baba4 --- /dev/null +++ b/untitled/contexture-client.md @@ -0,0 +1,4 @@ +# contexture-client + + + diff --git a/untitled/contexture-core.md b/untitled/contexture-core.md new file mode 100644 index 0000000..95cfe10 --- /dev/null +++ b/untitled/contexture-core.md @@ -0,0 +1,59 @@ +# contexture-core + +\# Contexture Core + +The core of Contexture is a package of its own. Located at [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture), it offers the very underlying function that is designed to process every one of the search queries. It begins with a simple curried function which, once the providers and the schemas are received, proceeds to process the Contexture DSL by walking down the given search tree, cleaning up every node of possible inconsistencies, then mutating the tree with the directions given by the provider types and schemas, up until a valid search query is obtained. This query is delivered to the provider `runSearch` method, and the result is finally added back to the tree. + +With this in mind, let's get some specifications. + +## Default Export + +Contexture's default export is a function that receives a total of three parameters, where the first two parameters are curried. + +The first argument is expected to be a plain JavaScript Object with two keys: + +* `providers`: Should be an object where each key will have a Contexture Provider. +* `schemas`: Should be an object where each key will have a Contexture Schema. + +Calling this function with this object only will return another function, which can be used as an asynchronous search runner. You can also pass in all the arguments as once, but the separation of parameters makes it easier to scope setting up the database providers, the types and the schemas from the search execution. + +Example declaration of a search function by passing the schema & providers object first: + +```javascript +const search = Contexture({ + schemas: { + ...mongoSchemas, + ...elasticSearchSchemas, + }, + providers: { + mongo: require('contexture-mongo')({ /* provider configuration */ }), + elasticsearch: require('contexture-elasticsearch')({ /* provider configuration */}), + }, +}) + +// How you might use it with an express-like API: +// app.use('/search', async req, res) => +// res.send(await search(req.body.search)) +``` + +The other two parameters are the search tree \(the Contexture DSL\), and an optional object that will be sent along to the provider's `runSearch` function as the first parameter, that can contain any property, but that should at least contain the following properties: + +| Option | Description | +| :--- | :--- | +| `debug` | Sends `_meta` as part of the response, which includes per node request records, relevant filters, and other debug info. | +| `onResult` | A callback which is called whenever a node finishes producing it's results, which can be used to send partial results over websockets for example. | + +This function, called at least up to the DSL search tree, will return a copy of the given search tree, filled with both properties needed to run the search, but also with the search results, which are assigned in the tree based on each one of the types that each specific search might be using. For more about how this happens, check out the Contexture Types. + +## The Algorithm + +_Coming soon..._ + +* Initial values in the tree. +* Flat lenses. +* State flags. +* Add filters. +* DFS initialization. + +## Utility Functions + diff --git a/untitled/contexture-dsl.md b/untitled/contexture-dsl.md new file mode 100644 index 0000000..8f43e22 --- /dev/null +++ b/untitled/contexture-dsl.md @@ -0,0 +1,31 @@ +# contexture-dsl + +\#\# Contexture DSL + +The Contexture DSL is a JavaScript object structure, or JSON structure, that is composed of nested nodes that are equal at all levels. Each node has a `key`, a `type` and many other optional properties. The first node is called the root and is a node of type `group`. Any node of type `group` can have many children. Each children can be a node of any type. If any children is another group, this children will probably have one or more other children nodes of any type, and so on. + +Let's begin talking about the root. + +## The Root + +The root of a Contexture Tree is a node of type `group`. Group types are required to have a `key`, the `type: "group"` property, and an array of children `children: [/* nodes */]`. Root groups also need an extra property: then ame of the schema. In summary, this is how a root node should look: + +```javascript +let searchTree = { + key: 'myRootNode', + type: 'group', + schema: 'mySchemaName', + children: [ + // Other nodes + ] +} +``` + +## The Children + +Each children will be an individual node. Each node will have at least a unique `key` \(unique per tree, but they can appear again in other trees\), and a `type`. Some types might require more properties. + +That's really all that it is for our DSL. The magic happens in the types and the providers. As long as your types are defined properly, and your providers build the correct queries and write them back in the tree, Contexture will work and you'll be able to use the Contexture DSL to build search interfaces of any complexity. + +**Note:** Many of the types have some common properties, like `field` or `values`. The only reason these properties are common is because they've made sense to be common for each one of our specific types, not because they have something to share between types. Each type has it's own properties and rules, so you should treat them as independent structures. + diff --git a/untitled/contexture-elasticsearch.md b/untitled/contexture-elasticsearch.md new file mode 100644 index 0000000..16d07e8 --- /dev/null +++ b/untitled/contexture-elasticsearch.md @@ -0,0 +1,4 @@ +# contexture-elasticsearch + +\# Contexture ElasticSearch + diff --git a/untitled/contexture-mongo.md b/untitled/contexture-mongo.md new file mode 100644 index 0000000..bcb3f39 --- /dev/null +++ b/untitled/contexture-mongo.md @@ -0,0 +1,4 @@ +# contexture-mongo + +\# Contexture Mongo + diff --git a/untitled/contexture-react.md b/untitled/contexture-react.md new file mode 100644 index 0000000..6681a09 --- /dev/null +++ b/untitled/contexture-react.md @@ -0,0 +1,4 @@ +# contexture-react + +\# Contexture React + diff --git a/untitled/design-principles.md b/untitled/design-principles.md new file mode 100644 index 0000000..aa6fdc1 --- /dev/null +++ b/untitled/design-principles.md @@ -0,0 +1,28 @@ +# design-principles + +\# Design Principles + +* Intentionally stateless. +* Detached from state management tools. +* Can work well with state management tools. +* Modern ES6+ +* Non-strict functional programming. +* Configuration based architecture. +* Focus on a very extensible small core. +* Small DSL. + * Database agnostic. + * Isomorphic Tree State. + * Optimized for database discovery. + * Optimized for advanced search interfaces. + * Aiming to be effective for arbitrarily complex database indexes. + * Simplicity over performance. +* Reaction management. + * Tree walking. + * State flags. + * What updates. + * Pauses. +* Why types? +* Why a results type? +* Why schemas? +* Why providers? + diff --git a/untitled/discovering-the-database.md b/untitled/discovering-the-database.md new file mode 100644 index 0000000..d409e95 --- /dev/null +++ b/untitled/discovering-the-database.md @@ -0,0 +1,89 @@ +# discovering-the-database + +\#\# Discovering The Database + +One of the great things about Contexture is that it can be used to discover the database. In this page, we'll see how to write a filter that not only allows the user to refine their search, but that also shows information about our data that might not be obvious by looking at the results directly. + +## The Facet Filter + +The `facet` type is just like any other type in the tree. It requires a unique key and some other properties. In contrast to previously seen types, such as `text` and `number`, facet doesn't require specific values, but instead it asks for two properties: `field`, and optionally `size`. This type is incredibly useful because besides filtering the results based on the `value` \(or `values`\) the user might have chosen, it retrieves from the database a list of available values! The `facet` type can be very well represented as a list of checkboxes, ready for users to pick for one or more values. Let's see how we can use it. + +## Adding the Facet Filter to the Tree + +To add the `facet` filter to the tree, we simply add it to the structure we already had. For example: + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'numberrange', + type: 'number', + field: 'age', + min: 0, + max: 100 + }, { + key: 'cityfacet', + type: 'facet', + field: 'city', + value: 'Orlando', // Optional + // We could have instead `values` with ['Orlando', 'Miami'] + }, { + key: 'results', + type: 'results' + }] +} +``` + +Once we have it in the tree, we can simply create another component to allow users to set a value to this type. In this case, however, we will be retrieving the available values from a field that is automatically inserted to this part of the tree. Let's make sure we have the data before we render the components. + +## Fetching the Available Options + +Having the facet node already added to the tree, it's only matter of running a first search before we render the components to actually get the possible results from the `facet` type. We can achieve this by calling to `contexture-client`'s `refresh` function: + +```javascript +// This would go after: +// let contextureSearchTree = Contexture(searchTree) +contextureSearchTree.refresh(['root']) +``` + +Now, the available options will be ready to be used at: `contextureSearchTree.getNode(['root', 'cityfacet']).context.options`. So we can write our facet component this way: + +```javascript +let toggleValue = (array, value) => { + if (array.indexOf(value) > -1) { + let copy = array.slice() // Making sure it's not a MobX Array + copy.splice(array.indexOf(value), 1) // removing value from the array + array.replace(copy) // MobX Arrays have this method to replace the array's inner values + } else { + array.push(value) + } +} +let RangeComponent = observer(({ tree }) => ( +
+ Select One or More: + {tree.getNode(['root', 'cityfacet']).context.options.map(option => ( +
+ -1} + onChange={() => toggleValue(tree.getNode(['root', 'cityfacet']).values, option.value)} + /> + +
+ ))} +
+)) +``` + +Including this component in our search interface will show the current available cities for the search results we have so far, and will allow users to filter the search even more by letting them pick any \(or many\) of the available cities. + diff --git a/untitled/diy-types.md b/untitled/diy-types.md new file mode 100644 index 0000000..368d7b2 --- /dev/null +++ b/untitled/diy-types.md @@ -0,0 +1,139 @@ +# diy-types + +\# DIY Types + +The Contexture ecosystem provides a defined list of types that can be used to perform a wide variety of different searches. Each type we offer also has a respective component in our `contexture-react` repo. We've made these components so you can quickstart your search interfaces! + +Even if our types are focused on the different search interfaces we provide, our API is designed to allow you to build any type you might need for any other possible use case you might encounter. + +We believe that making a generic framework will allow users to be creative on their search solutions. Because of that, we will start this document by explaining how to build your own types. + +Writing a new single type is about writing two plain JavaScript Objects: + +* One which is sent to the Contexture Provider. +* Another one which is sent to the initialization of the Context Tree. + +## How to Wite a Provider Type + +Initializing Contexture Core requires you to send a bunch of types per provider. A type on any provider is just a valid string property name on the object that is sent, accompanied with a corresponding value of a plain JavaScript Object with one or more of the following properties: + +| Property | Type | Params \(Type\) | Return Value Type | What it does | +| :--- | :--- | :--- | :--- | :--- | +| `hasValue` | Function | Node \(Object\) | Boolean | Allows Contexture to know wether or not to process this search. | +| `filter` | Function | Node \(Object\) | Object | Returns a query with the applied input values, which will be sent later to the database provider. | +| `result` | Function | Node \(Object\), Search \(Function\) | Promise | Allows running a direct search for this type before Comtexture sends the full seaech with the whole tree. | + +Once you have written a type, you can use it by sending it to an existing Contexture Provider. It should look more or less like: + +```javascript +let myType = { + hasValue: node => node.requiredProperty, + filter: node => ({ + providerQueryObject: { + value: node.requiredProperty + } + }) +} + +let provider = MyProvider({ + types: { + myType + } +}) +``` + +**Type names are not exclusive across providers**. You can define one type called `myAwesomeType` in more than one provider and you'll be able to keep the same required node properties, thus the same `hasValue`. This allows us to provide the same API for several types, and re-use code even if we switch the target database of the search. + +Once you have a type defined for one or more providers, you should write the same type for `contexture-client`. + +## How to Write a Client Type + +Contexture Client already provides a some types based on our `Example Types` \(more on that later.\) These type definitions help the client understand how a specific node affects every other node or itself. + +To create a custom type, you will need to think on the behaviors you might need for each one of the following properties: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `validate` | Function | Just as the Provider Type's `hasValue`, this function will let `contexture-client` know wether this node is valid for processing or not. | +| `reactors` | Object | The Reactors is how each of the node properties might affect this node or other nodes. See our Introduction to Reactors | +| `defaults` | Object | This object will help in the initialization of the nodes of the tree of this specific type through the definition of some default values on the specified properties. | + +More details about `contexture-client` types, their properties and their reserved words can be seen on the README of [contexture-client](https://github.com/smartprocure/contexture-client#client-types). + +The example types are already included in any instantiation of Contexture Client's Contexture Tree. However, you can add any type you need simply by extending the exposed `exampleTypes` with your own. In the following snippet, we initialize a `ContextureTree` with the available `exampleTypes`, and our new `myType`: + +```javascript +import * as ContextureClient from 'contexture-client' + +let tree = ContextureClient.ContextTree({ + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } +}, { + // Here we will have the underlying tree +}) +``` + +## How to Write a UI Component for a Type + +Writing a user interface for any type can be as simple as writing an HTML or JSX Element that will render or modify any property of the any node of an existing Contexture Tree, for example, using our custom type `myType`, we could write an input field that, onChange, will write the field's value into the `requiredProperty`. For example: + +```javascript +// This is ES6+ and JSX + +import * as ContextureClient from 'contexture-client' + +let tree = ContextureClient.ContextTree( + { + service: myService, + types: { + ...ContextureClient.exampleTypes, + myType: { + validate: node => node.requiredProperty, + reactors: { + requiredProperty: 'others', + }, + defaults: { + requiredProperty: false + } + } + } + }, { + key: 'root', + join: 'and', + children: [{ + key: 'myNode', + type: 'myType', + }] + } +) + +let node = tree.getNode(['root', 'myNode']) + +let Component = ({ node }) => ( + { + node.requiredProperty = e.target.value + }} + /> +) +``` + +Now that you have a component you can render it and play with it, but the component won't render by itself. If you want to see examples of custom components with automatic updates, please look at our Managing State Guide. + +## The Example Types + +With the intention of providing practical examples of how to write types, we decided to share some of the types we use in our production applications. These types belong to two different database processors: `contexture-elasticsearch` and `contexture-mongo`. + +**Example Types aren't the rule**. These types are only provided to serve as a guide to build any other type you might need for your application. You can also **extend our example types**, simply by assigning new types as properties on the objects exposed by `contexture-elasticsearch` and `contexture-mongo`. + diff --git a/untitled/elasticsearch-example-types.md b/untitled/elasticsearch-example-types.md new file mode 100644 index 0000000..cb76af2 --- /dev/null +++ b/untitled/elasticsearch-example-types.md @@ -0,0 +1,506 @@ +# elasticsearch-example-types + +\# ElasticSearch Example Types + +Contexture is designed to target any database you might need. However, so far we have only inplemented database providers for the only databases that we use: ElasticSearch and Mongo. + +Most of our types have relevant components already written to facilitate building search interfaces. As we progress over each one of these types, we will show small examples of simple components written to search with these types. We hope to provide enough information to allow you to take as little as you need, or as much as you want, and fulfill you expectations. + +Our ElasticSearch types are the following ones: + +## Results Type + +The `results` node is a node that if present, idicates to Contextre that the queries and aggregations should finally run and retrieve all the filtered values. It allows some customization properties, as shown below: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `page` | Number | Current page of results. | +| `pageSize` | Number | Total results per page. | +| `sortField` | String | Field used to sort the results. | +| `sortDir` | String | Direction of sorting \(`asc`, `desc`\) | + +## Bool Type + +The bool type is intended to work as an ElasticSearch terms aggregation with only one value for a single property. This is useful for user interfaces with a checkbox to include or exclude a specific field from a search \(or a specific field-value pair\). + +Here is the list of all properties this type uses: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| field | String | Name of the field that we will be using to filter the search search. | +| value | String | Value of the field that will be used to filter the search. | + +Example input: + +```javascript +{ + type: 'bool', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + term: { + fieldName: true + } +} +``` + +You can read more about it in: + +* [Source code of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/bool.js). +* [Unit tests of the type: bool](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/bool.js). +* Elastic Search [Term Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html). + +## Cardinality Type + +The Cardinality type serves to calculate an approximate count of the distinct available values. This type only uses one property, `field`. It returns it's values in the `context` of the node, rather than in the `results` node. + +Example input: + +```javascript +{ + type: 'cardinality', + field: 'Organization.Name.untouched' +} +``` + +Example output: + +```javascript +{ + aggs: { + cardinality: { + cardinality: { + field: 'Organization.Name.untouched' + } + } + } +} +``` + +You can read more about it in: + +* [Source code of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/cardinality.js). +* [Unit tests of the type: cardinality](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/cardinality.js). +* Elastic Search [Cardinality Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html). + +## Date Type + +The Date type is used to specify a range of dates that will be used to filter the available results. This range of dates can be specified by a string formatted date \(`YYYY-MM-DD`\) or by a small set of possible humanly readable date ranges. Details follow: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date \(formatted properly for ElasticSearch\), or one of the following values: `thisQuarter` \(for the current quarter of the year\), `lastQuarter` \(for the previously completed quarter of the year\), `nextQuarter` \(for the upcoming quarter\). | +| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date \(formatted properly for ElasticSearch\). | +| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | +| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | + +Example input 1: + +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25' +} +``` + +Example output 1: + +```javascript +{ + range: { + fieldName: { + gte: '2016-04-25', + format: 'dateOptionalTime' + } + } +} +``` + +Example input 2: + +```javascript +{ + type: 'date', + field: 'fieldName', + from: 'thisQuarter', + useDateMath: true, +} +``` + +Example output 2: + +```javascript +{ + range: { + fieldName: { + gte: moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD'), + lte: moment.utc(datemath.parse(`${moment().quarter(moment().quarter()).startOf('quarter').format('YYYY-MM-DD')}||+3M-1d/d`)).format('YYYY-MM-DD'), + format: 'dateOptionalTime' + } + } +} +``` + +You can read more about it in: + +* [Source code of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/date.js). +* [Unit tests of the type: date](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/date.js). +* Elastic Search [Range QUery](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html). + +## Date Histogram Type + +The `dateHistogram` type is very useful for retrieving the number of results within specific periods of time. The idea here is to end up building a chart showing how records have changed over time, for example by it's creation date or some other date property. This type is able to do this by running a nested stats aggregation inside a dateHistogram aggregation, while also supporting for tweaking min/max bounds. + +This type returns it's values in the `context` of the node, rather than in the `results` node. + +| Property Name | Type | Description | | +| :--- | :--- | :--- | :--- | +| `key_field` | String | What might be considered a good candidate for the X axis of the chart. | | +| `value_field` | String | What might be considered a good candidate for the Y axis of the chart. | | +| `interval` | | String | Available expressions for interval: `year` \(`1y`\), `quarter` \(`1q`\), `month` \(`1M`\), `week` \(`1w`\), `day` \(`1d`\), `hour` \(`1h`\), `minute` \(`1m`\), `second` \(`1s`\). | +| `boundsRange_min` | Date or String | Lower date limit that will be considered to filter the possible results. | | +| `boundsRange_max` | Date or String | Upper date limit that will be considered to filter the possible results. | | +| `boundsRange_useDateMath` | Boolean | If set to `true`, the min and max ranges will be valid as strings representative of dates, and will be formatted by NPM's library [`@elastic/datemath`](https://github.com/elastic/datemath-js). | | + +Example input: + +```javascript +{ + type: 'dateHistogram', + key_field: 'PO.IssuedDate', + value_field: 'LineItem.TotalPrice' +} +``` + +Example output: + +```javascript +{ + aggs: { + max_date: { + max: { + field: 'PO.IssuedDate', + }, + }, + min_date: { + min: { + field: 'PO.IssuedDate', + }, + }, + twoLevelAgg: { + date_histogram: { + field: 'PO.IssuedDate', + interval: 'year', + min_doc_count: 0, + }, + aggs: { + twoLevelAgg: { + stats: { + field: 'LineItem.TotalPrice', + }, + }, + }, + }, + }, +} +``` + +You can read more about it in: + +* [Source code of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateHistogram.js). +* [Unit tests of the type: dateHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/dateHistogram.js). +* [Two Level Aggregations in ElasticSearch](https://www.elastic.co/blog/intro-to-aggregations-pt-2-sub-aggregations). +* [Date Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html). + +## Exists Type + +The `exists` type is used to check wether some property exsits or not. It requires only two fields: `field` and `value`. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The target field we want to check. | +| `value` | Boolean | the value we want to check. Normally true or false. | + +Example input: + +```javascript +{ + type: 'exists', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + exists: { + field: 'fieldName' + } +} +``` + +You can read more about it in: + +* [Source code of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/exists.js). +* [Unit tests of the type: exists](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/exists.js). +* Elastic Search [Exists Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html). + +## Facet Type + +The `facet` type represents a list of dynamic choices, e.g. a checkbox list filter. We achieve this by running an ElasticSearch terms aggregation. We provide a way to limit the number of results that you will receive with the `size` property, so that large queries can safely be used. For that same purpose, the property `optionsFilter` is given, so that search queries can filter the results with a string. + +Facet returns it's values in the `context` of the node, rather than in the `results` node. + +| Property | Type | Default | Description | +| :--- | :--- | :--- | :--- | +| `field` | String | None, _required_ | The field it's operating on. | +| `mode` | String \(`include` or `exclude`\) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | +| `values` | Array \(of strings\) | `[]` | Already selected values. | +| `fieldMode` | String \(`autocomplete`, `word` or `suggest`\) | `autocomplete` | Whether to look at the entire field \(`autocomplete`\), the analyzed words in the field \(`word`\), or magic suggestions \(`suggest`\). | +| `size` | Number | 12 | How many options to return. | +| `cardinality` | Number | 5000 | Precision threshold override. | +| `includeZeroes` | Boolean | false | If true, it will include options with 0 matching documents \(aka `min_doc_count: 0`\) | +| `optionsFilter` | String | '' | Filters the options further, e.g. a find box above a checkbox list | +| `caseSensitive` | Boolean | false | Whether options filter is case sensitive. | +| `sort` | String \(`term` or `count`\) | `count` | Sort results alphabetically or by count of matching records. | + +Example input 1: + +```javascript +{ + type: 'facet', + field: 'fieldName', + values: ['abc', '123'] +} +``` + +Example output 1: + +```javascript +{ + terms: { + 'fieldName.untouched': ['abc', '123'] + } +} +``` + +Example input with exclude: + +```javascript +{ + type: 'facet', + field: 'fieldName', + mode: 'exclude', + values: ['abc', '123'], +} +``` + +Example output with exclude: + +```javascript +{ + bool: { + must_not: { + terms: { + 'fieldName.untouched': ['abc', '123'], + }, + }, + }, +} +``` + +You can read more about it in: + +* [Source code of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/facet.js). +* [Unit tests of the type: facet](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/facet.js). +* Elastic Search [Terms Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html). + +## Geo Type + +The `geo` type represents a geographic radius search. It needs a geocodeLocation service passed in to it. We currently assume that you will be using a google maps geocoder search. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String \(required\) | The field it's operating on | +| `location` | String \(required\) | Location to geocode \(e.g. an address, businessname, anything the google geocode can take\) | +| `radius` | Number \(required\) | Radius in miles | +| `operator` | String \(either `within` or `not within`\) | Whether the filter forces inclusion or exclusion \(defaults with `within`\). | + +Example input: + +```javascript +{ + type: 'geo', + field: 'fieldName', + location: 'SmartProcure', + radius: 10, + operator: 'within', +} +``` + +Example output: + +```javascript +{ + geo_distance: { + fieldName: '26.3170479,-80.1131784', + distance: '10mi', + }, +} +``` + +You can read more about it in: + +* [Source code of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/geo.js). +* [Unit tests of the type: geo](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/geo.js). +* [ElasticSearch's Geo Distance Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-query.html). + +## Number Type + +Number represents a number range with inclusive bounds. This type provides the ability to determine the best range values based on percentile interval and range threshold. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | + +Some Notes: 1. An empty value as the upper boundary represents infinity. 2. An empty value as the lower boundary represents negative infinity. 3. Zero has to be respected as a boundary value. 4. If findBestRange is true it will return the best min and max range. + +Example input: + +```javascript +{ + type: 'number', + field: 'fieldName', + min: 500, + max: 1000, +} +``` + +Example output: + +```javascript +{ + range: { + fieldName: { + gte: 500, + lte: 1000, + }, + }, +} +``` + +You can read more about it in: + +* [Source code of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/number.js). +* [Unit tests of the type: number](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/number.js). + +## Number Range Histogram Type + +The type `numberRangeHistogram` represents a number range with inclusive bounds. This type returns feedback in the form of histogram and statistical data. This type returns it's values in the `context` of the node, rather than in the `results` node. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | +| `percentileInterval` | Number | Used to group the results based on how many of the records are within each one of the sections given by the interval, from the `min` value, up to the `max` value. | + +Some Notes: 1. An empty value as the upper boundary represents infinity. 2. An empty value as the lower boundary represents negative infinity. 3. Zero has to be respected as a boundary value. + +* [Source code of the type: numberRangeHistogram](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/numberRangeHistohgram.js). +* [Histogram Aggregation in ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/search-aggregations-bucket-histogram-aggregation.html). + +## Query Type + +Query represents a raw elasticsearch query\_string. It's mostly used to provide a simple sarch box on the user interface. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The field we will be using to filter the results. | +| `query` | String | String that will be used to match the resulting records, based on the values available for the specified field on each one of the records. | +| `exact` | Boolean | Wether to match the query text as-is, or to try to be more flexible with the matches \(accepting values if they contain the given query, even if the casing doesn't match\). | + +Example input: + +```javascript +{ + type: 'query', + field: 'fieldName', + query: 'cable', + exact: true, +} +``` + +Example output: + +```javascript +{ + query_string: { + query: 'cable', + default_operator: 'AND', + default_field: 'fieldName.exact', + analyzer: 'exact', + }, +} +``` + +You can read more about it in: + +* [Source code of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/query.js). +* [Unit tests of the type: query](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/query.js). + +## Text Type + +Text implements raw text analysis like starts with, ends with, etc. These are generally regex queries. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The field we will be using to filter the results. | +| `join` | String | Either `any`, `all` or `none`. | +| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | +| `values` | Array | Array containing all the words that want to be used as inputs. | + +Example input: + +```javascript +{ + type: 'text', + field: 'fieldName', + join: 'any', + operator: 'contains', + values: ['laserjet', 'printer'], +} +``` + +Example output: + +```javascript +{ + query_string: { + default_field: 'fieldName', + default_operator: 'OR', + query: '"laserjet" "printer"', + }, +} +``` + +You can read more about it in: + +* [Source code of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/text.js). +* [Unit tests of the type: text](https://github.com/smartprocure/contexture-elasticsearch/blob/master/test/example-types/text.js). + +## Other ElasticSearch Example Types + +For more informaion about other available example types, please check: [https://github.com/smartprocure/contexture-elasticsearch](https://github.com/smartprocure/contexture-elasticsearch) + diff --git a/untitled/general.md b/untitled/general.md new file mode 100644 index 0000000..12ba9c9 --- /dev/null +++ b/untitled/general.md @@ -0,0 +1,77 @@ +# general + +\#\# Generally Useful Components + +## ContextureProvider + +This component is a magic tool to make Contexture search interfaces without having to write the tree separated. With this component, you can pass the tree directly as a plain object, and any children you pass will receive the `tree` property directly. + +This component receives: + +| Property Name | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `types` | Object | Yes | Your client-side types, such as the default types of the Contexture Client. | +| `service` | Object | Yes | The search function that sends the DSL to the initialized Contexture Core. | +| `nodeKey` | Object | Yes | Key for the root node. Defaults with `root`. | +| `...props` | Any other property \(children excluded\) | Yes | Any other property that you might want to send to the initialization of the Contexture Client. | + +**Note:** Any of the Contexture React Example Types components automatically add the nodes to the tree if the referenced node is not present. _This is an experimental feature_. + +Here's how you write your component: + +```javascript +let ContextureProvider = require('contexture-react/dist/ContextureProvider') +let ContextureClient = require('contexture-client') +let types = ContextureClient.exampleTypes +let service = async search => ({ + data: await postData('/sarch', { search }) +}) +// ... +// Later, on your render function, or where you put your components: + + + +``` + +* [Source code of the ContextureProvider component](https://github.com/smartprocure/contexture-react/blob/master/src/ContextureProvider.js). + +## FilterList + +A component that tries to automatically render the specific type components of the children of a node. + +| Property Name | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `node` | Object | Yes | Node of the Contexture Tree where the children that want to be rendered are. | +| `exampleTypes` | Object | No | Object which a key and a component per type. Defaults in the available example types components in contexture-elasticsearch. | +| `fields` | Object | Yes | Object which a key and an object with at least a `label` string property, used to indicate the label of each one of the filters. | + +Here's how you write your component: + +```javascript +let FilterList = require('contexture-react/dist/FilterList') +let ContextureClient = require('contexture-client') +let tree = ContextureClient({ + key: 'root', + type: 'group', + schema: 'mySchema', + children: [{ + key: 'query', + type: 'query', + field: 'myFieldName', + query: 'Something' + }] +}) +// ... +// Later, on your render function, or where you put your components: + +``` + +* [Source code of the FilterList component](https://github.com/smartprocure/contexture-react/blob/master/src/FilterList.js). + diff --git a/untitled/imdb-example.md b/untitled/imdb-example.md new file mode 100644 index 0000000..55fc56f --- /dev/null +++ b/untitled/imdb-example.md @@ -0,0 +1,49 @@ +# imdb-example + +\#\# IMDB Index + +In our Contexture React repository, we've created a Storybook live example that uses Contexture \(with no server whatsoever, everything in the client\) to query and discover a public ElasticSearch index with data similar to the data that IMDB might contain. + +* [Contexture React Repository](https://github.com/smartprocure/contexture-react). +* More information about [Storybook](https://github.com/storybooks/storybook). +* [The IMDB Live Index Explorer with Contexture](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). +* [The IMDB Live Index Explorer Source Code](https://smartprocure.github.io/contexture-react/?selectedKind=Index%20Explorer&selectedStory=Advanced%20Search). + +Let's see how we can use this tool. + +## Picking a Field + +The tool initially shows you a button named `Pick a Field`: + +![](https://i.imgur.com/CWz4Moy.png) + +This button will allow you to select any field on which you might want to run a search. If you click it, you'll see the following fields: + +![](https://i.imgur.com/RR6vMP1.png) + +If we pick `Actors`, for example, the search will trigger and results will appear: + +![](https://i.imgur.com/kep1LIo.png) + +Now, let's click on `Select Type` and click on `Facet`. A moment afterwards, we will get a list of checkboxes with the most common actors among the results: + +![](https://i.imgur.com/2Gq5FEh.png) + +We can see that the actor that has made the most movies \(among the ones idexed\) is Naveen Andrews. We can also see the number of movies this and other actors have made, and the total actors in the database \(of which we can only see 10 in this screenshot\). Under this list of actors, you'll see a text saying `View More`. If you click it, the next 10 actors will appear \(in order\). + +At the bottom of the page, you'll see a button labeled `Add Filter`. Clicking it will add another interactive component to the website, which will say `Click to add AND`. + +![](https://i.imgur.com/sRHN04n.png) + +Clicking the `Click to add AND` button will show us again a `Pick a Field` button: + +![](https://i.imgur.com/oA4LOEK.png) + +So we can start again. Let's say we pick the `Genre` field, and we click for the `Facet` type again. A bit later, the list of ordered genres will appear: + +![](https://i.imgur.com/nfbVlQ2.png) + +And that's it! We're discovering the database with a very simple and unpolished interface. By this point you might be curious on what components we're using to do all this magic. There's really no trick, just follow us through the tutorial. You can also skip some steps if you're particularly interested in: + +* [Contexture React Repository](https://github.com/smartprocure/contexture-react). + diff --git a/untitled/layout-components.md b/untitled/layout-components.md new file mode 100644 index 0000000..994b299 --- /dev/null +++ b/untitled/layout-components.md @@ -0,0 +1,115 @@ +# Layout Components + +## Awaiter + +Renders a loading indicator until a Promise is resolved. It will ender an exception \(currently just `Ooops...`\) if the Promise fails. if the Promise passes, the children are rendered. + +| Property Name | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `promise` | Promise | Yes | Minumum number for the range. | + +Here's how you write your component: + +```javascript +let Awaiter = require('contexture-react/dist/layout/Awaiter') +let promiseMaker = () => new Promise((resolve, reject)) /* ... */ resolve()) +// ... +// Later, on your render function, or where you put your components: + +
My Crazy Children
+ +``` + +* [Source code of the Awaiter component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/Awaiter.js). + +## BarChart + +Allows you to build your own bar charts \(besides just relying on the DateHistogram component\). + +| Property Name | Type | Default Value | Description | +| :--- | :--- | :--- | :--- | +| `valueField` | String | `''` | The field of each record where the value used for the height of each barnis located. | +| `categoryField` | String | `''` | The field of each record where the label of each bar is located. | +| `data` | Array \(of Objects with both the `valueField` and the `categoryField`\) | `[]` | The data that is going to be used to build the chart. | +| `height` | Number | `100` | Specifies the max height of the whole chart. | +| `format` | Function | `value => undefined` | Allows you to change the value that each one of the bars has. | +| `gutter` | Number | `5` | Allows you to specify the spacing between bars. | +| `yAxis` | Boolean | `false` | Allows you to specify wether you want Y axis information or not. | + +Here's how you write your component: + +```javascript +let BarChart = require('contexture-react/dist/layout/BarChart') +// ... +// Later, on your render function, or where you put your components: + +``` + +* [Source code of the BarChart component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/BarChart.js). + +## SpacedList + +Wraps every children in a div with a given style. Useful for \(as the name portrays\) making a spaced list. + +| Property Name | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `style` | Object | No | The style that will be applied to each div. Defaults in `{ marginBottom: '25px' }`. | + +Here's how you write your component: + +```javascript +let SpacedList = require('contexture-react/dist/layout/SpacedList') +// ... +// Later, on your render function, or where you put your components: + +

Hi

+ + +``` + +* [Source code of the SpacedList component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/SpacedList.js). + +## TextHighlight + +Used to highlight content within a text based on a pattern. + +| Property Name | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `pattern` | String | No | RegExp pattern used to find matching content. | +| `text` | String | Yes | Text used as the target of the matches. | +| `Wrap` | Component | No | Component used to wrap each one of the | + +matched. Defaults to `i`. \| + +Here's how you write your component: + +```javascript +let TextHighlight = require('contexture-react/dist/layout/TextHighlight') +// ... +// Later, on your render function, or where you put your components: + +``` + +* [Source code of the TextHighlight component](https://github.com/smartprocure/contexture-react/blob/master/src/layout/TextHighlight.js). + diff --git a/untitled/mongo-example-types.md b/untitled/mongo-example-types.md new file mode 100644 index 0000000..12b6b59 --- /dev/null +++ b/untitled/mongo-example-types.md @@ -0,0 +1,231 @@ +# mongo-example-types + +\# MongoDb Example Types + +Most of our types have relevant components already written to facilitate building search interfaces. As we progress over each one of these types, we will show small examples of simple components written to search with these types. We hope to provide enough information to allow you to take as little as you need, or as much as you want, and fulfill you expectations. + +Our MongoDb types are the following ones: + +## Results Type + +The `results` node is a node that if present, idicates to Contextre that the queries and aggregations should finally run and retrieve all the filtered values. It allows some customization properties, as shown below: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `page` | Number | Current page of results. | +| `pageSize` | Number | Total results per page. | +| `sortField` | String | Field used to sort the results. | +| `sortDir` | String | Direction of sorting \(`asc`, `desc`\) | + +## Date Type + +The Date type is used to specify a range of dates that will be used to filter the available results. This range of dates can be specified by a string formatted date \(`YYYY-MM-DD`\) or by a small set of possible humanly readable date ranges. Details follow: + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `from` | String | Either a date formatted as `YYYY-MM-DD`, or an ES date \(formatted properly for ElasticSearch\), or one of the following values: `thisQuarter` \(for the current quarter of the year\), `lastQuarter` \(for the previously completed quarter of the year\), `nextQuarter` \(for the upcoming quarter\). | +| `to` | String | Optional. Defaults to the current date. Only concidered if there's a `from` value, should be a date formatted as `YYYY-MM-DD` or an ES date \(formatted properly for ElasticSearch\). | +| `useDateMath` | Boolean | Defines if we should parse `from` as one of the `*Quarter` dates. | +| `isDateTime` | Boolean | Ignores any processing or formatting and accpets the `form` and `to` values as they come. | + +Example input: + +```javascript +{ + type: 'date', + field: 'fieldName', + from: '2016-04-25' +} +``` + +Example output: + +```javascript +{ + fieldName: { + $gte: new Date('2016-04-25'), + }, +} +``` + +You can read more about it in: + +* [Source code of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/date.js). +* [Unit tests of the type: date](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/date.js). +* MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). +* MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). + +## Exists Type + +The `exists` type is used to check wether some property exsits or not. It requires only two fields: `field` and `value`. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The target field we want to check. | +| `value` | Boolean | the value we want to check. Normally true or false. | + +Example input: + +```javascript +{ + type: 'exists', + field: 'fieldName', + value: true +} +``` + +Example output: + +```javascript +{ + $and: [ + { + fieldName: { + $exists: true, + $ne: '', + }, + }, + { + fieldName: { + $ne: null, + }, + }, + ], +} +``` + +You can read more about it in: + +* [Source code of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/exists.js). +* [Unit tests of the type: exists](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/exists.js). +* MongoDb's [$exists](https://docs.mongodb.com/manual/reference/operator/query/exists/). + +## Facet Type + +The `facet` type represents a list of dynamic choices, e.g. a checkbox list filter. We achieve this by running an ElasticSearch terms aggregation. We provide a way to limit the number of results that you will receive with the `size` property, so that large queries can safely be used. For that same purpose, the property `optionsFilter` is given, so that search queries can filter the results with a string. + +Facet returns it's values in the `context` of the node, rather than in the `results` node. + +| Property | Type | Default | Description | +| :--- | :--- | :--- | :--- | +| `field` | String | None, _required_ | The field it's operating on. | +| `mode` | String \(`include` or `exclude`\) | `include` | Wether this filter acts as for the inclusion or exclusion of the selected values when picking up what results to keep. | +| `values` | Array \(of strings\) | `[]` | Already selected values. | +| `size` | Number | 10 | How many options to return. | + +Example input: + +```javascript +{ + type: 'facet', + field: 'fieldName', + values: ['abc', '123'] +} +``` + +Example input with exclude: + +```javascript +{ + type: 'facet', + field: 'fieldName', + mode: 'exclude', + values: ['abc', '123'], +} +``` + +You can read more about it in: + +* [Source code of the type: facet](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js). + +## Number Type + +Number represents a number range with inclusive bounds. This type provides the ability to determine the best range values based on percentile interval and range threshold. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The field we will be using to filter the results. | +| `min` | Number | Lower boundary of the filter. | +| `max` | Number | Upper boundary of the filter. | + +Some Notes: 1. An empty value as the upper boundary represents infinity. 2. An empty value as the lower boundary represents negative infinity. 3. Zero has to be respected as a boundary value. 4. If findBestRange is true it will return the best min and max range. + +Example input: + +```javascript +{ + type: 'number', + field: 'fieldName', + min: 500, + max: 1000, +} +``` + +Example output: + +```javascript +{ + fieldName: { + $gte: 500, + $lte: 1000 + } +} +``` + +You can read more about it in: + +* [Source code of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/number.js). +* [Unit tests of the type: number](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/number.js). +* MongoDb's [$gte](https://docs.mongodb.com/manual/reference/operator/query/gte/). +* MongoDb's [$lte](https://docs.mongodb.com/manual/reference/operator/query/lte/). + +## Text Type + +Text implements raw text analysis like starts with, ends with, etc. These are generally regex queries. + +| Property Name | Type | Description | +| :--- | :--- | :--- | +| `field` | String | The field we will be using to filter the results. | +| `join` | String | Either `any`, `all` or `none`. | +| `operator` | String | `containsWord`, `startsWith`, `wordStartsWith`, `endsWith`, `wordEndsWith`, `is` or `containsExact`. | +| `values` | Array | Array containing all the words that want to be used as inputs. | + +Example input: + +```javascript +{ + type: 'text', + field: 'fieldName', + join: 'any', + operator: 'contains', + values: ['laserjet', 'printer'], +} +``` + +Example output: + +```javascript +{ + $or: [{ + fieldName: { + $regex: 'laserjet', + $options: 'i' + } + }, { + fieldName: { + $regex: 'printer', + $options: 'i' + } + }] +} +``` + +You can read more about it in: + +* [Source code of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/text.js). +* [Unit tests of the type: text](https://github.com/smartprocure/contexture-mongo/blob/master/test/example-types/text.js). + +## Other MongoDb Example Types + +For more informaion about other available example types, please check: [https://github.com/smartprocure/contexture-mongo](https://github.com/smartprocure/contexture-mongo) + diff --git a/untitled/reactors.md b/untitled/reactors.md new file mode 100644 index 0000000..499e335 --- /dev/null +++ b/untitled/reactors.md @@ -0,0 +1,24 @@ +# reactors + +\#\# Reactors + +When any node changes, depending on the type, it might be reasonable to re-trigger a search call for other nodes. We call this process the selection of `reactors`, where the possible reactors are only three: `self`, `others` and `all`. + +Reactors should be specified in the Client Types, where each type will have a specific reactor for each one of it's properties. For example \(taken from our client side [example types](https://github.com/smartprocure/contexture-client/blob/master/src/exampleTypes.js)\): + +```javascript +facet: { + reactors: { + values: 'others', + mode: 'others', + size: 'self', + optionsFilter: 'self', + sort: 'self', + } +} +``` + +The client side type defined above will be effective for any node with type `facet`, where the properties `values` and `mode` will affect only all the other nodes \(and not itself\), and the properties `size`, `optionsFilter` and `sort` will affect only the specific `facet` node and no other node. + +The one remaining reactor that isn't covered by that example is the `all` reactor. The difference between `others` and `all` is that `others` excludes the node where the change is happening, and `all` includes all other nodes and the node where the change is happening \(effectively combining `self` and `others`\). + diff --git a/untitled/simple-search-box.md b/untitled/simple-search-box.md new file mode 100644 index 0000000..767e675 --- /dev/null +++ b/untitled/simple-search-box.md @@ -0,0 +1,190 @@ +# simple-search-box + +\#\# Simple Search Box + +Building a simple search box consists of a single input field tied to any of the fields that any record might have in the specified database or index. To be able to make this text input tied to the search structure, we will need to bring `contexture-client` in. With that in mind, and the knowledge we already have of contexture, we can define the following tasks: + +1. Create a New Contexture Search Function. +2. Create a Web Server With a Search Endpoint +3. Write a Search Tree. +4. Make the Search Tree Aware of Contexture. +5. Write a Text Input. + +Let's dive in. + +## 1. Creating a New Conteture Search Function + +Just as how we saw in the previous pages, creating a new Contexture search function is about setting up the `contexture` package's default export with `schemas` and `providers`. In this case, we'll use the contexture-mongo approach in the following file \(let's call it `search.js`\): + +```javascript +let Contexture = require('contexture') +let provider = require('contexture-mongo') +let types = require('contexture-mongo/types') +let MongoClient = require('mongodb').MongoClient + +let schemas = { + collectionNameSchema: { + mongo: { + collection: 'collectionName' + } + } +} + +module.exports = {} + +MongoClient.connect('mongodb://localhost:27017', function(err, client) { + module.exports.search = Contexture({ + schemas, + providers: { + mongo: provider({ + getClient: () => client, + types: types() + }) + } + }) +}) +``` + +Note that we're now exporting the search function with `module.exports.search = Contexture(...`. You can also note that we specified the schema's name to be `collectionNameSchema`, and that any search using this schema will be using MongoDb's `collectionName` collection. + +## 2. Create a Web Server With a Search Endpoint + +If we want to separate the direct access to the database from the client, the previous code should live in the server. Our next step is to expose a `/search` endpoint so our future client can reach this function. In this section, we'll write a simple web server with `express` to satisfy our needs. + +### 2.1. Installing the Dependencies + +You'll need to install `express` and `body-parser` at the root of your project, as follows: + +```text +npm install --save express body-parser +``` + +### 2.2. Writing the Web Server + +Once you have the dependencies installed, you can set up the server with the following code: + +```javascript +let express = require('express') +let bodyParser = require('body-parser') +let search = require('./search') +let app = express() + +// create application/json parser +lete jsonParser = bodyParser.json() + +app.post('/search', jsonParser, (req, res) => { + if (!req.body || !req.body.search) return res.sendStatus(400) + search(req.body.search).then((err, result) => { + if (err) return res.send(401, err) + res.send(200, result) + }) +}) + +app.listen(3000) +``` + +You can read more about these topics in the following links: + +* [Express Documentation](https://expressjs.com/en/api.html). +* [body-parser repository](https://github.com/expressjs/body-parser). + +## 3. Writing a Search Tree + +Having the DSL processor available through a web server endpoint, we can follow up with the structure of the search interface itself. We'll conceptualize this by writing the Contexture DSL itself. + +Let's use the same `searchTree` that we used in our frist script: + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'results', + type: 'results' + }] +} +``` + +Keep in mind that we'll be using `collectionNameSchema` since we already defined a schema with that name on the server's `search.js` file. + +You can read more about writing Contexture DSL queries in our Querying documentation. + +## 4. Make the Search Tree Aware of Contexture + +Having a search tree, we will need `contexture-client` to make it smart enough for the user interface. Let's make sure we have it available in our project with: + +```text +npm install --save contexture-client +``` + +We will also use MobX to make it easier to notify our component that the state has changed. For that purpose, we will need to install `mobx` using NPM: + +```text +npm install --save mobx +``` + +Since we're heavy users of MobX, `contexture-client` already provides an adapter that we can use out of the box. Knowing this, let's prepare our `contexture-client` to work well with `mobx`, as follows: + +```javascript +let ContextureClient = require('contexture-client') +let ContextureMobx = require('contexture-react/dist/utils/contexture-mobx') + +let types = ContextureClient.exampleTypes +let service = async search => ({ + data: await postData('/sarch', { search }) +}) + +let Contexture = ContextureMobx({ + types, + service, +}) +``` + +Note that our service function will be the one responsible for sending the `searchTree` we previously defined to the DSL processor, we can wrap the search tree into a smart object that will later react to both the user input, and the search results. + +```javascript +let contextureSearchTree = Contexture(searchTree) +``` + +You can read more about these topics here: + +* [MobX](https://mobx.js.org/). +* [MobX Observers](https://mobx.js.org/refguide/observer-component.html). + +## 5. Writing a Text Input + +Having the search tree ready allows us to write a `mobx` observer component that will receive the tree and react to the result changes immediately. Here's an example: + +```text +let { observer } = require('mobx') +let SearchQuery = observer(({ tree }) => + { + tree.getNode(['root', 'namequery']).value = e.target.value + }} + /> + +let SearchResults = observer(({ tree }) => ( +
+ {JSON.stringify(tree.getNode(['root', 'results']).context.response.results)} +
+)) +``` + +Which we would render this way: + +```javascript + + +``` + +This will generate a text input that will trigger a new search every time the contents of the input change. This new search will get the results and show the results in a JSON form \(because of the `JSON.stringify` part\). Ideally, you will not render them in a JSON form, but render them using a list component or table. + diff --git a/untitled/your-first-filter.md b/untitled/your-first-filter.md new file mode 100644 index 0000000..fdf666a --- /dev/null +++ b/untitled/your-first-filter.md @@ -0,0 +1,60 @@ +# your-first-filter + +\#\# Your First Filter + +In the previous page, we wrote a simple search user interface where a single input would retrieve results. Now, we will add a simple filter that will allow us to get results within a given range. + +## Adding the Filter to the Tree + +The way we will add the filter is by adding a node to the tree. This node is going to have a type of `number`, which asks for a `field`, a `min` and a `max` values. + +```javascript +let searchTree = { + key: 'root', + type: 'group', + schema: 'collectionNameSchema', + children: [{ + key: 'namequery', + type: 'text', + field: 'name', + operator: 'containsWord', + value: 'text we want to match on the field name', + }, { + key: 'numberrange', + type: 'number', + field: 'age', + min: 0, + max: 100 + }, { + key: 'results', + type: 'results' + }] +} +``` + +With the `numberrange` node added, we can create another component with two inputs that will allow us to filter results with the range that the user specifies: + +```javascript +let RangeComponent = observer(({ tree }) => ( +
+ Minimum: + { + tree.getNode(['root', 'numberrange']).min = e.target.min + }} + /> +
+ Maximum: + { + tree.getNode(['root', 'numberrange']).max = e.target.max + }} + /> +
+)) +``` + +And that's it! Rendering this component will make our search aware of any change the user might desire on the minimum and maximum ages they might want to use to filter the available results. + From 549570fbb2e74fa50f884b383a7fdbb18fc92a6d Mon Sep 17 00:00:00 2001 From: me Date: Wed, 11 Mar 2020 04:38:31 +0000 Subject: [PATCH 148/150] GitBook: [master] 43 pages modified --- SUMMARY.md | 48 +++++++------ .../connecting-other-databases.md | 0 .../discovering-the-database.md | 0 {untitled => getting-started}/imdb-example.md | 0 .../simple-search-box.md | 0 .../your-first-filter.md | 0 {untitled => other-components}/general.md | 0 .../layout-components.md | 0 {untitled => querying}/contexture-dsl.md | 0 .../diy-types.md | 0 .../elasticsearch-example-types.md | 0 .../mongo-example-types.md | 0 types-and-type-components/untitled.md | 2 - .../building-your-own-provider.md | 0 .../contexture-client.md | 0 .../contexture-core.md | 0 .../contexture-elasticsearch.md | 0 .../contexture-mongo.md | 0 .../contexture-react.md | 0 .../design-principles.md | 0 {untitled => under-the-hood}/reactors.md | 0 {querying => under-the-hood}/untitled.md | 0 untitled/README.md | 72 ------------------- 23 files changed, 26 insertions(+), 96 deletions(-) rename {untitled => getting-started}/connecting-other-databases.md (100%) rename {untitled => getting-started}/discovering-the-database.md (100%) rename {untitled => getting-started}/imdb-example.md (100%) rename {untitled => getting-started}/simple-search-box.md (100%) rename {untitled => getting-started}/your-first-filter.md (100%) rename {untitled => other-components}/general.md (100%) rename {untitled => other-components}/layout-components.md (100%) rename {untitled => querying}/contexture-dsl.md (100%) rename {untitled => types-and-type-components}/diy-types.md (100%) rename {untitled => types-and-type-components}/elasticsearch-example-types.md (100%) rename {untitled => types-and-type-components}/mongo-example-types.md (100%) delete mode 100644 types-and-type-components/untitled.md rename {untitled => under-the-hood}/building-your-own-provider.md (100%) rename {untitled => under-the-hood}/contexture-client.md (100%) rename {untitled => under-the-hood}/contexture-core.md (100%) rename {untitled => under-the-hood}/contexture-elasticsearch.md (100%) rename {untitled => under-the-hood}/contexture-mongo.md (100%) rename {untitled => under-the-hood}/contexture-react.md (100%) rename {untitled => under-the-hood}/design-principles.md (100%) rename {untitled => under-the-hood}/reactors.md (100%) rename {querying => under-the-hood}/untitled.md (100%) delete mode 100644 untitled/README.md diff --git a/SUMMARY.md b/SUMMARY.md index c2547c5..8b8cb83 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -13,32 +13,36 @@ * [Project Setup](getting-started/project-setup.md) * [Your First Contexture Script](getting-started/your-first-contexture-script.md) * [Connecting to Elasticsearch & MongoDB](getting-started/connecting-to-elasticsearch-and-mongodb.md) +* [connecting-other-databases](getting-started/connecting-other-databases.md) +* [simple-search-box](getting-started/simple-search-box.md) +* [your-first-filter](getting-started/your-first-filter.md) +* [discovering-the-database](getting-started/discovering-the-database.md) +* [imdb-example](getting-started/imdb-example.md) ## Querying -* [Untitled](querying/untitled.md) +* [contexture-dsl](querying/contexture-dsl.md) ## Types and Type Components -* [Untitled](types-and-type-components/untitled.md) -* [README](untitled/README.md) - * [building-your-own-provider](untitled/building-your-own-provider.md) - * [connecting-other-databases](untitled/connecting-other-databases.md) - * [contexture-client](untitled/contexture-client.md) - * [contexture-core](untitled/contexture-core.md) - * [contexture-dsl](untitled/contexture-dsl.md) - * [contexture-elasticsearch](untitled/contexture-elasticsearch.md) - * [contexture-mongo](untitled/contexture-mongo.md) - * [contexture-react](untitled/contexture-react.md) - * [design-principles](untitled/design-principles.md) - * [discovering-the-database](untitled/discovering-the-database.md) - * [diy-types](untitled/diy-types.md) - * [elasticsearch-example-types](untitled/elasticsearch-example-types.md) - * [general](untitled/general.md) - * [imdb-example](untitled/imdb-example.md) - * [Layout Components](untitled/layout-components.md) - * [mongo-example-types](untitled/mongo-example-types.md) - * [reactors](untitled/reactors.md) - * [simple-search-box](untitled/simple-search-box.md) - * [your-first-filter](untitled/your-first-filter.md) +* [diy-types](types-and-type-components/diy-types.md) +* [elasticsearch-example-types](types-and-type-components/elasticsearch-example-types.md) +* [mongo-example-types](types-and-type-components/mongo-example-types.md) + +## Other Components + +* [general](other-components/general.md) +* [Layout Components](other-components/layout-components.md) + +## Under the Hood + +* [Untitled](under-the-hood/untitled.md) +* [design-principles](under-the-hood/design-principles.md) +* [contexture-core](under-the-hood/contexture-core.md) +* [contexture-elasticsearch](under-the-hood/contexture-elasticsearch.md) +* [contexture-mongo](under-the-hood/contexture-mongo.md) +* [building-your-own-provider](under-the-hood/building-your-own-provider.md) +* [contexture-client](under-the-hood/contexture-client.md) +* [reactors](under-the-hood/reactors.md) +* [contexture-react](under-the-hood/contexture-react.md) diff --git a/untitled/connecting-other-databases.md b/getting-started/connecting-other-databases.md similarity index 100% rename from untitled/connecting-other-databases.md rename to getting-started/connecting-other-databases.md diff --git a/untitled/discovering-the-database.md b/getting-started/discovering-the-database.md similarity index 100% rename from untitled/discovering-the-database.md rename to getting-started/discovering-the-database.md diff --git a/untitled/imdb-example.md b/getting-started/imdb-example.md similarity index 100% rename from untitled/imdb-example.md rename to getting-started/imdb-example.md diff --git a/untitled/simple-search-box.md b/getting-started/simple-search-box.md similarity index 100% rename from untitled/simple-search-box.md rename to getting-started/simple-search-box.md diff --git a/untitled/your-first-filter.md b/getting-started/your-first-filter.md similarity index 100% rename from untitled/your-first-filter.md rename to getting-started/your-first-filter.md diff --git a/untitled/general.md b/other-components/general.md similarity index 100% rename from untitled/general.md rename to other-components/general.md diff --git a/untitled/layout-components.md b/other-components/layout-components.md similarity index 100% rename from untitled/layout-components.md rename to other-components/layout-components.md diff --git a/untitled/contexture-dsl.md b/querying/contexture-dsl.md similarity index 100% rename from untitled/contexture-dsl.md rename to querying/contexture-dsl.md diff --git a/untitled/diy-types.md b/types-and-type-components/diy-types.md similarity index 100% rename from untitled/diy-types.md rename to types-and-type-components/diy-types.md diff --git a/untitled/elasticsearch-example-types.md b/types-and-type-components/elasticsearch-example-types.md similarity index 100% rename from untitled/elasticsearch-example-types.md rename to types-and-type-components/elasticsearch-example-types.md diff --git a/untitled/mongo-example-types.md b/types-and-type-components/mongo-example-types.md similarity index 100% rename from untitled/mongo-example-types.md rename to types-and-type-components/mongo-example-types.md diff --git a/types-and-type-components/untitled.md b/types-and-type-components/untitled.md deleted file mode 100644 index 5094080..0000000 --- a/types-and-type-components/untitled.md +++ /dev/null @@ -1,2 +0,0 @@ -# Untitled - diff --git a/untitled/building-your-own-provider.md b/under-the-hood/building-your-own-provider.md similarity index 100% rename from untitled/building-your-own-provider.md rename to under-the-hood/building-your-own-provider.md diff --git a/untitled/contexture-client.md b/under-the-hood/contexture-client.md similarity index 100% rename from untitled/contexture-client.md rename to under-the-hood/contexture-client.md diff --git a/untitled/contexture-core.md b/under-the-hood/contexture-core.md similarity index 100% rename from untitled/contexture-core.md rename to under-the-hood/contexture-core.md diff --git a/untitled/contexture-elasticsearch.md b/under-the-hood/contexture-elasticsearch.md similarity index 100% rename from untitled/contexture-elasticsearch.md rename to under-the-hood/contexture-elasticsearch.md diff --git a/untitled/contexture-mongo.md b/under-the-hood/contexture-mongo.md similarity index 100% rename from untitled/contexture-mongo.md rename to under-the-hood/contexture-mongo.md diff --git a/untitled/contexture-react.md b/under-the-hood/contexture-react.md similarity index 100% rename from untitled/contexture-react.md rename to under-the-hood/contexture-react.md diff --git a/untitled/design-principles.md b/under-the-hood/design-principles.md similarity index 100% rename from untitled/design-principles.md rename to under-the-hood/design-principles.md diff --git a/untitled/reactors.md b/under-the-hood/reactors.md similarity index 100% rename from untitled/reactors.md rename to under-the-hood/reactors.md diff --git a/querying/untitled.md b/under-the-hood/untitled.md similarity index 100% rename from querying/untitled.md rename to under-the-hood/untitled.md diff --git a/untitled/README.md b/untitled/README.md deleted file mode 100644 index c3c4b0a..0000000 --- a/untitled/README.md +++ /dev/null @@ -1,72 +0,0 @@ -# README - -\# Contexture Documentation - -This repo is designed to host the documentation of Contexture as a whole. - -## Table of Contents - - - -* [Getting Started]() - * [Connecting to Other Databases](connecting-other-databases.md) - * [Simple Search Box](simple-search-box.md) - * [Your First Filter](your-first-filter.md) - * [Discovering the Database](discovering-the-database.md) - * [IMDB Index](imdb-example.md) -* [Querying]() - * [Contexture DSL](contexture-dsl.md) - * [Interactive Queries]() - * [Contexture Client](contexture-client.md) - * [Initializing the Contexture Client](contexture-client.md#initializing-the-contexture-client) - * [Context Tree](contexture-client.md#context-tree) - * [Tree and getNode](contexture-client.md#tree-and-getnode) - * [Mutate, Add and Remove](contexture-client.md#mutate-add-and-remove) - * [Introduction to Reactors](reactors.md) -* [Types and Type Components]() - * [DIY Types](diy-types.md) - * [How to Write a Provider Type](diy-types.md#how-to-wite-a-provider-type) - * [How to Write a Client Type](diy-types.md#how-to-wite-a-client-type) - * [How to Write a UI Component for a Type](diy-types.md#how-to-write-a-ui-component-for-a-type) - * [The Example Types](diy-types.md#the-example-types) - * [ElasticSearch Example Types](elasticsearch-example-types.md) - * [Mongo Example Types](mongo-example-types.md) - * [Available React Components for Types]() -* [Other Components]() - * [Generally Useful Components](general.md) - * [Layout Components](layout-components.md) -* [Managing State]() - * [DIY State Management](docs/diy.md) - * [MobX](docs/mobx.md) - * [Redux \(Coming Soon\)](docs/redux.md) -* [Theming]() -* [Recommendations]() - * [Architecture](docs/architecture.md) - * [Client vs Server](docs/architecture.md#client-vs-server) - * [Type definitions](docs/architecture.md#type-definitions) - * [Scaling](docs/architecture.md#scaling) - * [Server Side Searches](docs/server-side-searches.md) - * [Searching On an Endpoint](docs/server-side-searches.md#searching-on-an-endpoint) - * [Caching](docs/server-side-searches.md#caching) - * [Client Side Searches](docs/client-side-searches.md) - * [Click to Search](docs/client-side-searches.md#click-to-search) - * [Real Time Searches](docs/client-side-searches.md#real-time-searches) - * [Cascading](docs/client-side-searches.md#cascading) -* [Under the Hood]() - * [Design Principles](design-principles.md) - * [Contexture Core](contexture-core.md) - * [Default Export](contexture-core.md#default-export) - * [The Algorithm](contexture-core.md#the-algorithm) - * [Utility Functions](contexture-core.md#utility-functions) - * [Contexture Providers]() - * [Contexture ElasticSearch]() - * [Contexture Mongo]() - * [Building Your Own Provider](building-your-own-provider.md) - * [Contexture Client](contexture-client.md) - * [State Properties](contexture-client.md#state-properties) - * [Lenses](contexture-client.md#lenses) - * [Reactors in Detail](contexture-client.md#reactors-in-detail) - * [Serialization](contexture-client.md#serialization) - * [Traversals](contexture-client.md#traversals) - * [Contexture React](contexture-react.md) - From 99354c51c870cb4bda40777f34bd29a8b8385d1c Mon Sep 17 00:00:00 2001 From: me Date: Wed, 11 Mar 2020 04:53:33 +0000 Subject: [PATCH 149/150] GitBook: [master] 24 pages modified --- SUMMARY.md | 37 +++++++++---------- ...es.md => connecting-to-other-databases.md} | 4 +- getting-started/discovering-the-database.md | 4 +- .../{imdb-example.md => imdb-index.md} | 4 +- getting-started/simple-search-box.md | 4 +- getting-started/your-first-filter.md | 4 +- ...eral.md => generally-useful-components.md} | 4 +- querying/contexture-dsl.md | 4 +- types-and-type-components/diy-types.md | 4 +- .../elasticsearch-example-types.md | 4 +- ...mple-types.md => mongodb-example-types.md} | 4 +- under-the-hood/building-your-own-provider.md | 4 +- under-the-hood/contexture-client.md | 4 +- under-the-hood/contexture-core.md | 4 +- under-the-hood/contexture-elasticsearch.md | 4 +- under-the-hood/contexture-mongo.md | 4 +- under-the-hood/contexture-react.md | 4 +- under-the-hood/design-principles.md | 4 +- under-the-hood/reactors.md | 4 +- under-the-hood/untitled.md | 2 - 20 files changed, 40 insertions(+), 71 deletions(-) rename getting-started/{connecting-other-databases.md => connecting-to-other-databases.md} (78%) rename getting-started/{imdb-example.md => imdb-index.md} (98%) rename other-components/{general.md => generally-useful-components.md} (98%) rename types-and-type-components/{mongo-example-types.md => mongodb-example-types.md} (99%) delete mode 100644 under-the-hood/untitled.md diff --git a/SUMMARY.md b/SUMMARY.md index 8b8cb83..ba4c965 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -13,36 +13,35 @@ * [Project Setup](getting-started/project-setup.md) * [Your First Contexture Script](getting-started/your-first-contexture-script.md) * [Connecting to Elasticsearch & MongoDB](getting-started/connecting-to-elasticsearch-and-mongodb.md) -* [connecting-other-databases](getting-started/connecting-other-databases.md) -* [simple-search-box](getting-started/simple-search-box.md) -* [your-first-filter](getting-started/your-first-filter.md) -* [discovering-the-database](getting-started/discovering-the-database.md) -* [imdb-example](getting-started/imdb-example.md) +* [Connecting to Other Databases](getting-started/connecting-to-other-databases.md) +* [Simple Search Box](getting-started/simple-search-box.md) +* [Your First Filter](getting-started/your-first-filter.md) +* [Discovering The Database](getting-started/discovering-the-database.md) +* [IMDB Index](getting-started/imdb-index.md) ## Querying -* [contexture-dsl](querying/contexture-dsl.md) +* [Contexture DSL](querying/contexture-dsl.md) ## Types and Type Components -* [diy-types](types-and-type-components/diy-types.md) -* [elasticsearch-example-types](types-and-type-components/elasticsearch-example-types.md) -* [mongo-example-types](types-and-type-components/mongo-example-types.md) +* [DIY Types](types-and-type-components/diy-types.md) +* [ElasticSearch Example Types](types-and-type-components/elasticsearch-example-types.md) +* [MongoDB Example Types](types-and-type-components/mongodb-example-types.md) ## Other Components -* [general](other-components/general.md) +* [Generally Useful Components](other-components/generally-useful-components.md) * [Layout Components](other-components/layout-components.md) ## Under the Hood -* [Untitled](under-the-hood/untitled.md) -* [design-principles](under-the-hood/design-principles.md) -* [contexture-core](under-the-hood/contexture-core.md) -* [contexture-elasticsearch](under-the-hood/contexture-elasticsearch.md) -* [contexture-mongo](under-the-hood/contexture-mongo.md) -* [building-your-own-provider](under-the-hood/building-your-own-provider.md) -* [contexture-client](under-the-hood/contexture-client.md) -* [reactors](under-the-hood/reactors.md) -* [contexture-react](under-the-hood/contexture-react.md) +* [Design Principles](under-the-hood/design-principles.md) +* [Contexture Core](under-the-hood/contexture-core.md) +* [Contexture ElasticSearch](under-the-hood/contexture-elasticsearch.md) +* [Contexture Mongo](under-the-hood/contexture-mongo.md) +* [Building Your Own Provider](under-the-hood/building-your-own-provider.md) +* [Contexture Client](under-the-hood/contexture-client.md) +* [Reactors](under-the-hood/reactors.md) +* [Contexture React](under-the-hood/contexture-react.md) diff --git a/getting-started/connecting-other-databases.md b/getting-started/connecting-to-other-databases.md similarity index 78% rename from getting-started/connecting-other-databases.md rename to getting-started/connecting-to-other-databases.md index 7194332..8da837d 100644 --- a/getting-started/connecting-other-databases.md +++ b/getting-started/connecting-to-other-databases.md @@ -1,6 +1,4 @@ -# connecting-other-databases - -\#\# Connecting to Other Databases +# Connecting to Other Databases Contexture depends on it's providers to be able to know how to translate from the Contexture DSL to the specific DSL that each database needs. Because of this, to connect to other databases you will need to create a new Provider. diff --git a/getting-started/discovering-the-database.md b/getting-started/discovering-the-database.md index d409e95..e9ac3b9 100644 --- a/getting-started/discovering-the-database.md +++ b/getting-started/discovering-the-database.md @@ -1,6 +1,4 @@ -# discovering-the-database - -\#\# Discovering The Database +# Discovering The Database One of the great things about Contexture is that it can be used to discover the database. In this page, we'll see how to write a filter that not only allows the user to refine their search, but that also shows information about our data that might not be obvious by looking at the results directly. diff --git a/getting-started/imdb-example.md b/getting-started/imdb-index.md similarity index 98% rename from getting-started/imdb-example.md rename to getting-started/imdb-index.md index 55fc56f..3469780 100644 --- a/getting-started/imdb-example.md +++ b/getting-started/imdb-index.md @@ -1,6 +1,4 @@ -# imdb-example - -\#\# IMDB Index +# IMDB Index In our Contexture React repository, we've created a Storybook live example that uses Contexture \(with no server whatsoever, everything in the client\) to query and discover a public ElasticSearch index with data similar to the data that IMDB might contain. diff --git a/getting-started/simple-search-box.md b/getting-started/simple-search-box.md index 767e675..95a4d85 100644 --- a/getting-started/simple-search-box.md +++ b/getting-started/simple-search-box.md @@ -1,6 +1,4 @@ -# simple-search-box - -\#\# Simple Search Box +# Simple Search Box Building a simple search box consists of a single input field tied to any of the fields that any record might have in the specified database or index. To be able to make this text input tied to the search structure, we will need to bring `contexture-client` in. With that in mind, and the knowledge we already have of contexture, we can define the following tasks: diff --git a/getting-started/your-first-filter.md b/getting-started/your-first-filter.md index fdf666a..c6aa621 100644 --- a/getting-started/your-first-filter.md +++ b/getting-started/your-first-filter.md @@ -1,6 +1,4 @@ -# your-first-filter - -\#\# Your First Filter +# Your First Filter In the previous page, we wrote a simple search user interface where a single input would retrieve results. Now, we will add a simple filter that will allow us to get results within a given range. diff --git a/other-components/general.md b/other-components/generally-useful-components.md similarity index 98% rename from other-components/general.md rename to other-components/generally-useful-components.md index 12ba9c9..42a8082 100644 --- a/other-components/general.md +++ b/other-components/generally-useful-components.md @@ -1,6 +1,4 @@ -# general - -\#\# Generally Useful Components +# Generally Useful Components ## ContextureProvider diff --git a/querying/contexture-dsl.md b/querying/contexture-dsl.md index 8f43e22..0dce262 100644 --- a/querying/contexture-dsl.md +++ b/querying/contexture-dsl.md @@ -1,6 +1,4 @@ -# contexture-dsl - -\#\# Contexture DSL +# Contexture DSL The Contexture DSL is a JavaScript object structure, or JSON structure, that is composed of nested nodes that are equal at all levels. Each node has a `key`, a `type` and many other optional properties. The first node is called the root and is a node of type `group`. Any node of type `group` can have many children. Each children can be a node of any type. If any children is another group, this children will probably have one or more other children nodes of any type, and so on. diff --git a/types-and-type-components/diy-types.md b/types-and-type-components/diy-types.md index 368d7b2..22bc3a8 100644 --- a/types-and-type-components/diy-types.md +++ b/types-and-type-components/diy-types.md @@ -1,6 +1,4 @@ -# diy-types - -\# DIY Types +# DIY Types The Contexture ecosystem provides a defined list of types that can be used to perform a wide variety of different searches. Each type we offer also has a respective component in our `contexture-react` repo. We've made these components so you can quickstart your search interfaces! diff --git a/types-and-type-components/elasticsearch-example-types.md b/types-and-type-components/elasticsearch-example-types.md index cb76af2..306c6ae 100644 --- a/types-and-type-components/elasticsearch-example-types.md +++ b/types-and-type-components/elasticsearch-example-types.md @@ -1,6 +1,4 @@ -# elasticsearch-example-types - -\# ElasticSearch Example Types +# ElasticSearch Example Types Contexture is designed to target any database you might need. However, so far we have only inplemented database providers for the only databases that we use: ElasticSearch and Mongo. diff --git a/types-and-type-components/mongo-example-types.md b/types-and-type-components/mongodb-example-types.md similarity index 99% rename from types-and-type-components/mongo-example-types.md rename to types-and-type-components/mongodb-example-types.md index 12b6b59..f018281 100644 --- a/types-and-type-components/mongo-example-types.md +++ b/types-and-type-components/mongodb-example-types.md @@ -1,6 +1,4 @@ -# mongo-example-types - -\# MongoDb Example Types +# MongoDB Example Types Most of our types have relevant components already written to facilitate building search interfaces. As we progress over each one of these types, we will show small examples of simple components written to search with these types. We hope to provide enough information to allow you to take as little as you need, or as much as you want, and fulfill you expectations. diff --git a/under-the-hood/building-your-own-provider.md b/under-the-hood/building-your-own-provider.md index 52b2b8c..dfab9ae 100644 --- a/under-the-hood/building-your-own-provider.md +++ b/under-the-hood/building-your-own-provider.md @@ -1,6 +1,4 @@ -# building-your-own-provider - -\# Building Your Own Provider +# Building Your Own Provider Writing a new provider consists of writing a function \(or class or any other approach you might like\), that will receive a configuration object with at least the following properties: diff --git a/under-the-hood/contexture-client.md b/under-the-hood/contexture-client.md index 13baba4..50da32f 100644 --- a/under-the-hood/contexture-client.md +++ b/under-the-hood/contexture-client.md @@ -1,4 +1,4 @@ -# contexture-client - +# Contexture Client +TODO diff --git a/under-the-hood/contexture-core.md b/under-the-hood/contexture-core.md index 95cfe10..9034401 100644 --- a/under-the-hood/contexture-core.md +++ b/under-the-hood/contexture-core.md @@ -1,6 +1,4 @@ -# contexture-core - -\# Contexture Core +# Contexture Core The core of Contexture is a package of its own. Located at [github.com/smartprocure/contexture](https://github.com/smartprocure/contexture), it offers the very underlying function that is designed to process every one of the search queries. It begins with a simple curried function which, once the providers and the schemas are received, proceeds to process the Contexture DSL by walking down the given search tree, cleaning up every node of possible inconsistencies, then mutating the tree with the directions given by the provider types and schemas, up until a valid search query is obtained. This query is delivered to the provider `runSearch` method, and the result is finally added back to the tree. diff --git a/under-the-hood/contexture-elasticsearch.md b/under-the-hood/contexture-elasticsearch.md index 16d07e8..0b63342 100644 --- a/under-the-hood/contexture-elasticsearch.md +++ b/under-the-hood/contexture-elasticsearch.md @@ -1,4 +1,4 @@ -# contexture-elasticsearch +# Contexture ElasticSearch -\# Contexture ElasticSearch +TODO diff --git a/under-the-hood/contexture-mongo.md b/under-the-hood/contexture-mongo.md index bcb3f39..2f58d0d 100644 --- a/under-the-hood/contexture-mongo.md +++ b/under-the-hood/contexture-mongo.md @@ -1,4 +1,4 @@ -# contexture-mongo +# Contexture Mongo -\# Contexture Mongo +TODO diff --git a/under-the-hood/contexture-react.md b/under-the-hood/contexture-react.md index 6681a09..b4e78b6 100644 --- a/under-the-hood/contexture-react.md +++ b/under-the-hood/contexture-react.md @@ -1,4 +1,4 @@ -# contexture-react +# Contexture React -\# Contexture React +TODO diff --git a/under-the-hood/design-principles.md b/under-the-hood/design-principles.md index aa6fdc1..b94c6e2 100644 --- a/under-the-hood/design-principles.md +++ b/under-the-hood/design-principles.md @@ -1,6 +1,4 @@ -# design-principles - -\# Design Principles +# Design Principles * Intentionally stateless. * Detached from state management tools. diff --git a/under-the-hood/reactors.md b/under-the-hood/reactors.md index 499e335..11a7f0e 100644 --- a/under-the-hood/reactors.md +++ b/under-the-hood/reactors.md @@ -1,6 +1,4 @@ -# reactors - -\#\# Reactors +# Reactors When any node changes, depending on the type, it might be reasonable to re-trigger a search call for other nodes. We call this process the selection of `reactors`, where the possible reactors are only three: `self`, `others` and `all`. diff --git a/under-the-hood/untitled.md b/under-the-hood/untitled.md deleted file mode 100644 index 5094080..0000000 --- a/under-the-hood/untitled.md +++ /dev/null @@ -1,2 +0,0 @@ -# Untitled - From b5aa39ea2eb172348d37abec449377980b3e1eec Mon Sep 17 00:00:00 2001 From: me Date: Wed, 11 Mar 2020 04:57:41 +0000 Subject: [PATCH 150/150] GitBook: [master] 3 pages modified --- SUMMARY.md | 2 ++ about-contexture/alternatives-and-benchmarks.md | 4 ++++ about-contexture/brief-history.md | 4 ++++ 3 files changed, 10 insertions(+) create mode 100644 about-contexture/alternatives-and-benchmarks.md create mode 100644 about-contexture/brief-history.md diff --git a/SUMMARY.md b/SUMMARY.md index ba4c965..eb2c259 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -7,6 +7,8 @@ * [What Is Contexture](about-contexture/what-is-contexture.md) * [Glossary of Terms](about-contexture/glossary-of-terms.md) * [Map of Repos](about-contexture/map-of-repos.md) +* [Brief History](about-contexture/brief-history.md) +* [Alternatives & Benchmarks](about-contexture/alternatives-and-benchmarks.md) ## Getting Started diff --git a/about-contexture/alternatives-and-benchmarks.md b/about-contexture/alternatives-and-benchmarks.md new file mode 100644 index 0000000..de6a452 --- /dev/null +++ b/about-contexture/alternatives-and-benchmarks.md @@ -0,0 +1,4 @@ +# Alternatives & Benchmarks + +TODO + diff --git a/about-contexture/brief-history.md b/about-contexture/brief-history.md new file mode 100644 index 0000000..1da679a --- /dev/null +++ b/about-contexture/brief-history.md @@ -0,0 +1,4 @@ +# Brief History + +TODO +