Custom NLP Providers

Abstractor has the ability to outsource the generation of suggestions against narrative text to an external NLP provider. The communication between Abstractor and the custom NLP provider is done over REST API calls.

Here are the steps necessary to make this happen:

• Run the rake task 'bundle exec rake abstractor:custom_nlp_provider' in the host application. This will create a YAML configuration at 'config/abstractor/custom_nlp_providers.yml'. The YAML file will contain a suggestion endpoint URL for each configured custom NLP provider for each environment:

sample_custom_nlp_provider:
  suggestion_endpoint:
      development: http://custom-nlp-provider.dev/suggest
      test: http://custom-nlp-provider.test/suggest
      staging: http://custom-nlp-provider-staging.org/suggest
      production: http://custom-nlp-provider.org/suggest

• Configure Abstractor to use the custom NLP provide to generate suggestions,

• Abstractor expose a resource listing abstraction schemas: /abstractor_abstraction_schemas/

• Abstractor will expose a resource exposing the details of an abstraction schema: /abstractor_abstraction_schemas/1/. This resource will expose the schema's predicate, display name, object type, preferred name, object values and object value variants.

• During the abstraction process, if an abstraction source has a source type of 'custom nlp suggestion' , Abstractor will use the name of the provider in the custom_nlp_provider column and look up the endpoint in its current environment. Abstractor will post to the provider's endpoint a unique identifier of the abstraction schema, a unique identifier of the abstraction, a unique identifier of the abstraction source and the textual content. Abstractor might pass object values and object value variants if we decide synchronization between the external library and Abstractor is too much overhead.

• Here is a rough draft of the JSON message that will be posted by Abstractor to the provider's configured endpoint:

{
  "abstractor_abstraction_schema_id":1,
  "abstractor_abstraction_id":1,
  "abstractor_abstraction_source_id":1,
  "source_type":  "PathologyCase",
  "source_method": "note_text",
  "text": "The patient has a diagnosis of glioblastoma.  GBM does not have a good prognosis.  But I can't rule out meningioma.",
  "object_values": [
    { "value": "glioblastoma, nos",
      "object_value_variants":[
        { "value": "glioblastoma" },
        { "value": "gbm" },
        { "value": "spongioblastoma multiforme"}
      ]
    },
    { "value": "meningioma, nos",
      "object_value_variants":[
        { "value": "meningioma" },
        { "value": "leptomeningioma" },
        { "value": "meningeal fibroblastoma" }
      ]
    }
  ]
}

• The provider will be expected to generate suggestions and invoke an endpoint provided by Abstractor to insert a suggestion and its suggestion sources: /abstractor_abstractions/1/abstractor_suggestions/. The endpoint will expect a post with a JSON message in the following format:

{
   "abstractor_suggestion":{
      "abstractor_abstraction_source_id":1,
      "source_id":1,
      "source_type": "PathologyCase",
      "source_method": "note_text",
      "value":"glioblastoma",
      "unknown":null,
      "not_applicable":null,
      "suggestion_sources":[
                  {
                  "match_value":"glioblastoma",
                  "sentence_match_value":"The patient has a diagnosis of glioblastoma."
               },
               {
                  "match_value":"gbm",
                  "sentence_match_value":"GBM does not have a good prognosis."
               }
            ]
   }
}

• If the provider wants to make more than one suggestion, then it will need to invoke the endpoint again with a new suggestion:

{
   "abstractor_suggestion":{
      "abstractor_abstraction_source_id":1,
      "source_id":1,
      "source_type": "PathologyCase",
      "source_method": "note_text",
      "value":"meningioma",
      "unknown":null,
      "not_applicable":null,
      "sources":[
         {
            "match_value":"meningioma",
            "sentence_match_value":"But I can't rule out meningioma."
         }
      ]
   }
}