diff --git a/deepsearch/model/README.md b/deepsearch/model/README.md index 2d7c7580..e69de29b 100644 --- a/deepsearch/model/README.md +++ b/deepsearch/model/README.md @@ -1,83 +0,0 @@ -# Model API -> Currently in **beta**. - -The Model API is a unified and extensible inference API across different model kinds. - -Built-in model kind support includes NLP annotators and QA generators. - -## Installation -To use the Model API, install including the `api` extra, i.e.: -- with poetry: -`poetry add "deepsearch-toolkit[api]"` -- with pip: `pip install "deepsearch-toolkit[api]"` - -## Basic usage -```python -from deepsearch.model.server.config import Settings -from deepsearch.model.server.model_app import ModelApp - -# (1) create an app -app = ModelApp(settings=Settings()) - -# (2) register your model(s) -model = ... # e.g. SimpleGeoNLPAnnotator() -app.register_model(model) - -# (3) run the app -app.run(host="127.0.0.1", port=8000) -``` - -### Settings -App configuration is done in [`Settings`](server/config.py) based on -[Pydantic Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings/). - -E.g. the required API key can be injected via env var `DS_MODEL_API_KEY`. - -### OpenAPI -The OpenAPI UI is served under `/docs`, i.e. by default at http://127.0.0.1:8000/docs. - -## Developing a new model -To develop a new model class for an existing [kind](kinds/), inherit from the base model -class of that kind and implement the abstract methods and attributes. - -The framework will automatically use the correct controller for your model. - -To use a custom controller instead, pass it to `ModelApp.register_model()` via the -optional parameter `controller`. - -### Examples -- [Dummy NLP annotator](examples/dummy_nlp_annotator/) -- [Simple geo NLP annotator](examples/simple_geo_nlp_annotator/) -- [Dummy QA generator](examples/dummy_qa_generator/) - -Note: these examples configure the app with API key "example123"; when running them, use -the same key to access the protected endpoints. - -### Inference -As as example, an input payload for the `/predict` endpoint for the geography annotator -could look as follows (note that `deepsearch.res.ibm.com/x-deadline` should be a -future timestamp): -```json -{ - "apiVersion": "v1", - "kind": "NLPModel", - "metadata": { - "annotations": { - "deepsearch.res.ibm.com/x-deadline": "2024-04-20T12:26:01.479484+00:00", - "deepsearch.res.ibm.com/x-transaction-id": "abc", - "deepsearch.res.ibm.com/x-attempt-number": 5, - "deepsearch.res.ibm.com/x-max-attempts": 5 - } - }, - "spec": { - "findEntities": { - "entityNames": ["cities", "countries"], - "objectType": "text", - "texts": [ - "Bern, the capital city of Switzerland, is built around a crook in the Aare River.", - "Athens is a major coastal urban area in the Mediterranean and is both the capital and largest city of Greece." - ] - } - } -} -``` diff --git a/deepsearch/model/examples/dummy_nlp_annotator/README.MD b/deepsearch/model/examples/dummy_nlp_annotator/README.md similarity index 100% rename from deepsearch/model/examples/dummy_nlp_annotator/README.MD rename to deepsearch/model/examples/dummy_nlp_annotator/README.md diff --git a/deepsearch/model/examples/dummy_qa_generator/README.MD b/deepsearch/model/examples/dummy_qa_generator/README.md similarity index 100% rename from deepsearch/model/examples/dummy_qa_generator/README.MD rename to deepsearch/model/examples/dummy_qa_generator/README.md diff --git a/deepsearch/model/examples/simple_geo_nlp_annotator/README.MD b/deepsearch/model/examples/simple_geo_nlp_annotator/README.md similarity index 100% rename from deepsearch/model/examples/simple_geo_nlp_annotator/README.MD rename to deepsearch/model/examples/simple_geo_nlp_annotator/README.md diff --git a/docs/guide/index.md b/docs/guide/index.md index 65073eaf..97be0dc8 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -18,4 +18,4 @@ - [Custom CLI plugins](./cli_plugins.md) ## Custom models -- [Custom model examples](./model.md) +- [Model API](./model.md) \ No newline at end of file diff --git a/docs/guide/model.md b/docs/guide/model.md index 35b21893..32d70fe2 100644 --- a/docs/guide/model.md +++ b/docs/guide/model.md @@ -1,10 +1,17 @@ -## Launching a model +# Model API +> Currently in **beta**. -To run this example make sure you've installed the full environment including the optional installs provided in poetry +The Model API is a unified and extensible inference API across different model kinds. - poetry install --all-extras +Built-in model kind support includes NLP annotators and QA generators. -Then run the model with: +## Installation +To use the Model API, install including the `api` extra, i.e.: +- with poetry: +`poetry add "deepsearch-toolkit[api]"` +- with pip: `pip install "deepsearch-toolkit[api]"` + +To launch a model: python -m deepsearch.model.examples.. @@ -35,347 +42,350 @@ this API key must be provided on the authorization header for most application e - /model/{model_name}/predict - You can make POST requests to have the model annotate your data, refer to the [Sample Requests](#Sample NLP kind models requests and responses) - /health - An endpoint that will respond with a preset message letting you know that the webserver is healthy. -### Annotator API endpoints guide +### OpenAPI + +The OpenAPI UI is served under `/docs`, e.g. http://127.0.0.1:8000/docs. -You can direcly access the API via a browser to the provided url on the console upon running the application, usually: +## Developing a new model +To develop a new model class for an existing [kind](kinds/), inherit from the base model +class of that kind and implement the abstract methods and attributes. - http://127.0.0.1:8000 -This will take you to the landing page. Here you will likely find that you are not authenticated, however you can still check if the API is responsive by accessing the /health endpoint +The framework will automatically use the correct controller for your model. - http://127.0.0.1:8000/health -It will be easier to interact with the application prediction capabilities via the provided documentation endpoint +To use a custom controller instead, pass it to `ModelApp.register_model()` via the +optional parameter `controller`. - http://127.0.0.1:8000/docs +### Examples +- [Dummy NLP annotator](examples/dummy_nlp_annotator/) +- [Simple geo NLP annotator](examples/simple_geo_nlp_annotator/) +- [Dummy QA generator](examples/dummy_qa_generator/) ## Sample NLP kind models requests and responses ### Entity annotation - -```json - { - "apiVersion": "string", - "kind": "NLPModel", - "metadata": { - "annotations": { - "deepsearch.res.ibm.com/x-deadline": "2038-01-18T00:00:00.000Z", - "deepsearch.res.ibm.com/x-transaction-id": "string", - "deepsearch.res.ibm.com/x-attempt-number": "string", - "deepsearch.res.ibm.com/x-max-attempts": "string" - } - }, - "spec": { - "findEntities": { - "entityNames": ["entity_foo", "entity_bar"], - "objectType": "text", - "texts": [ - "A piece of text", - "Yet another piece of text" - ] - } - } - } -``` - -response - -```json -{ - "entities":[ - { - "entity_foo":[ - { - "type":"entity_foo", - "match":"a 'entity_foo' match in 'A piece of text'", - "original":"a 'entity_foo' original in 'A piece of text'", - "range":[ - 1, - 5 - ] - }, - { - "type":"entity_foo", - "match":"another 'entity_foo' match in 'A piece of text'", - "original":"another 'entity_foo' original in 'A piece of text'", - "range":[ - 12, - 42 - ] - } - ], - "entity_bar":[ - { - "type":"entity_bar", - "match":"a 'entity_bar' match in 'A piece of text'", - "original":"a 'entity_bar' original in 'A piece of text'", - "range":[ - 1, - 5 - ] - }, - { - "type":"entity_bar", - "match":"another 'entity_bar' match in 'A piece of text'", - "original":"another 'entity_bar' original in 'A piece of text'", - "range":[ - 12, - 42 - ] - } - ] - }, - { - "entity_foo":[ - { - "type":"entity_foo", - "match":"a 'entity_foo' match in 'Yet another piece of text'", - "original":"a 'entity_foo' original in 'Yet another piece of text'", - "range":[ - 1, - 5 - ] - }, - { - "type":"entity_foo", - "match":"another 'entity_foo' match in 'Yet another piece of text'", - "original":"another 'entity_foo' original in 'Yet another piece of text'", - "range":[ - 12, - 42 - ] - } - ], - "entity_bar":[ - { - "type":"entity_bar", - "match":"a 'entity_bar' match in 'Yet another piece of text'", - "original":"a 'entity_bar' original in 'Yet another piece of text'", - "range":[ - 1, - 5 - ] - }, - { - "type":"entity_bar", - "match":"another 'entity_bar' match in 'Yet another piece of text'", - "original":"another 'entity_bar' original in 'Yet another piece of text'", - "range":[ - 12, - 42 - ] - } - ] - } - ] -} -``` - -### Relationship annotation -request -```json -{ - "apiVersion":"string", - "kind":"NLPModel", - "metadata":{ - "annotations":{ - "deepsearch.res.ibm.com/x-deadline":"2038-01-18T00:00:00.000Z", - "deepsearch.res.ibm.com/x-transaction-id":"string", - "deepsearch.res.ibm.com/x-attempt-number":"string", - "deepsearch.res.ibm.com/x-max-attempts":"string" - } - }, - "spec":{ - "findRelationships":{ - "relationshipNames": null, - "objectType":"text", - "texts":[ - "Lisbon, Madrid, Paris and Zurich are Capitals of european countries", - "Berlin is the capital of Germany" - ], - "entities":[ - { - "cities":[ - { - "type":"cities", - "match":"Lisbon", - "original":"Lisbon", - "range":[ - 0, - 6 - ] - }, - { - "type":"cities", - "match":"Madrid", - "original":"Madrid", - "range":[ - 8, - 14 - ] - }, - { - "type":"cities", - "match":"Paris", - "original":"Paris", - "range":[ - 16, - 21 - ] - } - ], - "countries":[ - - ] +??? note "Entity annotation request payload" + ```json + { + "apiVersion": "string", + "kind": "NLPModel", + "metadata": { + "annotations": { + "deepsearch.res.ibm.com/x-deadline": "2038-01-18T00:00:00.000Z", + "deepsearch.res.ibm.com/x-transaction-id": "string", + "deepsearch.res.ibm.com/x-attempt-number": "string", + "deepsearch.res.ibm.com/x-max-attempts": "string" + } }, - { - "cities":[ - { - "type":"cities", - "match":"Berlin", - "original":"Berlin", - "range":[ - 0, - 6 - ] - } - ], - "countries":[ - { - "type":"countries", - "match":"Germany", - "original":"Germany", - "range":[ - 25, - 32 - ] - } - ] + "spec": { + "findEntities": { + "entityNames": ["entity_foo", "entity_bar"], + "objectType": "text", + "texts": [ + "A piece of text", + "Yet another piece of text" + ] + } } - ] - } - } -} -``` + } + ``` -response +??? note "Entity annotation request response" + ```json + { + "entities":[ + { + "entity_foo":[ + { + "type":"entity_foo", + "match":"a 'entity_foo' match in 'A piece of text'", + "original":"a 'entity_foo' original in 'A piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_foo", + "match":"another 'entity_foo' match in 'A piece of text'", + "original":"another 'entity_foo' original in 'A piece of text'", + "range":[ + 12, + 42 + ] + } + ], + "entity_bar":[ + { + "type":"entity_bar", + "match":"a 'entity_bar' match in 'A piece of text'", + "original":"a 'entity_bar' original in 'A piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_bar", + "match":"another 'entity_bar' match in 'A piece of text'", + "original":"another 'entity_bar' original in 'A piece of text'", + "range":[ + 12, + 42 + ] + } + ] + }, + { + "entity_foo":[ + { + "type":"entity_foo", + "match":"a 'entity_foo' match in 'Yet another piece of text'", + "original":"a 'entity_foo' original in 'Yet another piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_foo", + "match":"another 'entity_foo' match in 'Yet another piece of text'", + "original":"another 'entity_foo' original in 'Yet another piece of text'", + "range":[ + 12, + 42 + ] + } + ], + "entity_bar":[ + { + "type":"entity_bar", + "match":"a 'entity_bar' match in 'Yet another piece of text'", + "original":"a 'entity_bar' original in 'Yet another piece of text'", + "range":[ + 1, + 5 + ] + }, + { + "type":"entity_bar", + "match":"another 'entity_bar' match in 'Yet another piece of text'", + "original":"another 'entity_bar' original in 'Yet another piece of text'", + "range":[ + 12, + 42 + ] + } + ] + } + ] + } + ``` -```json -{ - "relationships": [ +### Relationship annotation +??? note "Relationship annotation request payload" + ```json { - "cities-to-countries": { - "header": [ - "cities", - "countries", - "weight", - "source" - ], - "data": [] - }, - "cities-to-provincies": { - "header": [ - "cities", - "provincies", - "weight", - "source" - ], - "data": [] - }, - "provincies-to-countries": { - "header": [ - "provincies", - "countries", - "weight", - "source" - ], - "data": [] - } - }, + "apiVersion":"string", + "kind":"NLPModel", + "metadata":{ + "annotations":{ + "deepsearch.res.ibm.com/x-deadline":"2038-01-18T00:00:00.000Z", + "deepsearch.res.ibm.com/x-transaction-id":"string", + "deepsearch.res.ibm.com/x-attempt-number":"string", + "deepsearch.res.ibm.com/x-max-attempts":"string" + } + }, + "spec":{ + "findRelationships":{ + "relationshipNames": null, + "objectType":"text", + "texts":[ + "Lisbon, Madrid, Paris and Zurich are Capitals of european countries", + "Berlin is the capital of Germany" + ], + "entities":[ + { + "cities":[ + { + "type":"cities", + "match":"Lisbon", + "original":"Lisbon", + "range":[ + 0, + 6 + ] + }, + { + "type":"cities", + "match":"Madrid", + "original":"Madrid", + "range":[ + 8, + 14 + ] + }, + { + "type":"cities", + "match":"Paris", + "original":"Paris", + "range":[ + 16, + 21 + ] + } + ], + "countries":[ + + ] + }, + { + "cities":[ + { + "type":"cities", + "match":"Berlin", + "original":"Berlin", + "range":[ + 0, + 6 + ] + } + ], + "countries":[ + { + "type":"countries", + "match":"Germany", + "original":"Germany", + "range":[ + 25, + 32 + ] + } + ] + } + ] + } + } + } + ``` + +??? note "Relationship annotation request response" + ```json { - "cities-to-countries": { - "header": [ - "cities", - "countries", - "weight", - "source" - ], - "data": [ - [ - "cities.0", - "countries.0", - 1, - "entities" - ] - ] - }, - "cities-to-provincies": { - "header": [ - "cities", - "provincies", - "weight", - "source" - ], - "data": [] - }, - "provincies-to-countries": { - "header": [ - "provincies", - "countries", - "weight", - "source" - ], - "data": [] - } + "relationships": [ + { + "cities-to-countries": { + "header": [ + "cities", + "countries", + "weight", + "source" + ], + "data": [] + }, + "cities-to-provincies": { + "header": [ + "cities", + "provincies", + "weight", + "source" + ], + "data": [] + }, + "provincies-to-countries": { + "header": [ + "provincies", + "countries", + "weight", + "source" + ], + "data": [] + } + }, + { + "cities-to-countries": { + "header": [ + "cities", + "countries", + "weight", + "source" + ], + "data": [ + [ + "cities.0", + "countries.0", + 1, + "entities" + ] + ] + }, + "cities-to-provincies": { + "header": [ + "cities", + "provincies", + "weight", + "source" + ], + "data": [] + }, + "provincies-to-countries": { + "header": [ + "provincies", + "countries", + "weight", + "source" + ], + "data": [] + } + } + ] } - ] -} -``` + ``` ### Property annotation - + TBD ## Sample QAGen kind models requests and responses ### Generate -Request -```json -{ - "apiVersion": "string", - "kind": "QAGenModel", - "metadata": { - "annotations": { - "deepsearch.res.ibm.com/x-deadline": "2038-01-18T00:00:00.000Z", - "deepsearch.res.ibm.com/x-transaction-id": "string", - "deepsearch.res.ibm.com/x-attempt-number": "string", - "deepsearch.res.ibm.com/x-max-attempts": "string" - } - }, - "spec": { - "generateAnswers": { - "contexts": [ - ["What is the best model"] - ], - "questions": [ - "If you are a dummy repeat what I said!" - ] +??? note "Genarate request payload" + ```json + { + "apiVersion": "string", + "kind": "QAGenModel", + "metadata": { + "annotations": { + "deepsearch.res.ibm.com/x-deadline": "2038-01-18T00:00:00.000Z", + "deepsearch.res.ibm.com/x-transaction-id": "string", + "deepsearch.res.ibm.com/x-attempt-number": "string", + "deepsearch.res.ibm.com/x-max-attempts": "string" + } + }, + "spec": { + "generateAnswers": { + "contexts": [ + ["What is the best model"] + ], + "questions": [ + "If you are a dummy repeat what I said!" + ] + } + } } - } -} -``` - -Response + ``` -```json -{ - "answers": [ - "If you are a dummy repeat what I said!" - ] -} -``` +??? note "Generate request response" + ```json + { + "answers": [ + "If you are a dummy repeat what I said!" + ] + } + ``` ## Important considerations - Each annotator has a kind, for example NLPModel, as such the kind for the request must match. - For NLP Kind annotators under the spec you must define the appropriate types to be annotated, for the dummyNLPAnnotator [refer to this example](#marker1) you will find on the request that we would like to find *entity_foo* and *entity_bar* an empty list will lead to -no annotations being made, a null object will lead to *all* possible annotations being made. -- Each annotator declared what sort of input it supports, a list constituted of any number of (text, table and image). +no annotations being made, a null object will lead to **all** possible annotations being made. +- Each annotator declares what sort of input it supports, a list constituted of any number of (text, table and image). - The x-deadline on each request is already implemented and must lie some time in the future. - refer to the /docs page on any annotator instance for more specification on the request types \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index f0dbcb98..96e692c7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -60,7 +60,7 @@ nav: - Knowledge graphs: guide/kgs.md - APIs: guide/apis.md - Plugin system: guide/cli_plugins.md - - Custom model examples: guide/model.md + - Model API: guide/model.md - Example gallery: gallery/index.md - API reference: - Toolkit reference: api-reference.md