Versions Compared

Old Version 6

changes.mady.by.user Andrew McMurry

Saved on

compared with

New Version Current

changes.mady.by.user Genoa Polumbo

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3
Table of Contents

Reference Docs

Info

Venn Diagram

Image Added

Feature Set Construction (Text words-> Lexical Features)

Office Excel
namescrubber_classification_table.xls
Info

3.X is a new vision for the scrubber. As we approached diminishing returns for improving REGEX and whitelists/black lists, we have shifted towards a machine learning methods approach and learning from large bodies of medical information from publications and UMLS dictionaries.

3.X Diagram In Progress Image Removed

Table of Contents

Use Case: Tagging Noun Phrases and UMLS concepts

Precondition:

  • Training Data: Genia, PenTree Bank, Mayo Source
  • Software: cTakes using features POS tagger & UMLS CUID extractor

Steps:

  1. Block of text is sent to cTakes
  2. cTakes processing
    1. start & end position of all POS tags
    2. part of speech
      1. Most interested in Nouns because of PHI
    3. Need Info: are cUIDS associated with WORDS or PHRASES?

Post-condition:

  • Input document (either medical note OR publication) will have POS tagged and UMLS CUIDs.

Use Case: Meta-analysis of text

Precondition:

  • Tagging Noun Phrases
  • Scubber configured (with or without local dictionary/regex mods)

Steps:

  1. Each "scrubber" implementation procudes Recorder output
    1. Passthrough Imp
      1. Regex
      2. Word lists
    2. cTakes Impl (OpenNLP)
      1. Noun Phrases
      2. UMLS cuids
  2. Performance evaluation (ROC)
    1. Scrubber standalone
    2. Scrubber word lists limited by detected noun phrases
    3. Scrubber word lists limited by detected noun phrases and non-UMLS concepts

Post-Condition

...