Skip to main content
Back to NLP Topics The Fundamental Meaning (FM) engine is a deterministic, rule-based NLP model built on ChatScript. It analyzes word meaning, position, conjugation, and sentence structure to identify intent.

​
How the FM Engine Works

​
Processing Pipeline

StepDescription
TokenizationSplits text into tokens. Retains hyphenated words; splits digit-hyphen combos (e.g., 2-3 → 2 – 3).
SubstitutionExpands abbreviations, contractions, and misspellings using system concepts. Bot substitutions apply custom replacements (e.g., Chevy → Chevrolet).
MergingCombines sequences of single words, numbers, or dates into single tokens (e.g., credit card, twenty-five).
Spell CheckReplaces unknown words with known matches using WordNet and app-defined terms.
LemmatizationConverts words to their base form using WordNet.
GleaningIdentifies and normalizes special segments: polite phrases (treated as noise), multi-intent connectors, numbers, and system entities (dates, currencies, phone numbers).
POS TaggingTags each word with its part of speech and role (subject/verb/object).
MarkingAssigns a concept/meaning to each word based on context. Example: book in “book a flight” is marked as a verb.

​
Scoring

The FM engine scores each intent match based on:
  • Position of matched words in the sentence.
  • Whether matched words are nouns or verbs.
  • Role: main subject, main verb, or main object.
  • Exact match vs. synonym match.
  • Tense: present/future tense is preferred.
The engine prefers tasks where matching words are close together, toward the start of the sentence, and in the same order as the task label.

​
Key Components

ComponentDescription
SynonymsAlternative words for task/entity names. App-wide.
ConceptsPredefined or custom groups of related terms. Used in patterns (e.g., ~world_country, ~fruit).
PatternsSentence structures for idiomatic or non-obvious expressions. Pattern matches take priority over synonym matches.

​
Manage Synonyms

How synonyms work: Synonyms for a word apply across all tasks and entities that contain that word. Add Bot Synonyms:
  1. Go to Natural Language > Training > Bot Synonyms.
  2. Click New Synonym.
  3. Enter the keyword and its synonyms (press Enter after each).
Add Entity Synonyms:
  1. Go to Natural Language > Training > Entities.
  2. Click the Edit icon for an entity.
  3. Enter synonyms and press Enter.
Parallel editing: Multiple users can add or delete synonyms concurrently. The platform tracks add/delete operations individually—no data is overwritten.

​
Manage Concepts

System concepts: Built-in groups (e.g., ~yes, ~no, ~emohello). Extend instead of modifying: create a new concept and add words. Custom concepts:
  1. Go to Natural Language > Training > Concepts > New Concept.
  2. Name must start with ~ followed by alphanumerics and underscores.
  3. Add concept members (one per line).
Use concepts in patterns: buy ~fruit matches buy orange, buy banana, etc. Disable default help: Add ~disable_help concept and train.

​
App Substitutions

Replaces words in user utterances before NLP processing. Define using the ~bot_substitutions concept:
  • Syntax: original_word|replacement
  • Case-insensitive match; case-sensitive replacement.
  • When replacement is empty (no text after |), the word is removed.
Special patterns:
  • ?_word|replacement — matches a number followed by the word: ?_km|kilometers → 5 km becomes 5 kilometers.
  • < word|replacement — matches only when the word appears first in the sentence.

​
App Word Aliases

Aliases normalize STT transcription errors using the + prefix on Bot Synonyms. Example: +Alpha Romeo as an alias for Alfa Romeo — corrects voice transcription to the canonical form. Aliases do not change the original utterance but:
  • Return the normalized value for Custom Concept entities.
  • Link concepts to new words (Marking).

​
Homophones

Handle words that sound alike but are spelled differently (e.g., write / right). Define using a Concept with members separated by |: 
write|right|rite
The engine considers all homophones during POS tagging:
  • If a homophone is a better grammatical fit, it replaces the original word.
  • If equally valid, the original word is marked with the homophone.

​
Manage Patterns and Rules

See NLP Guidelines — Patterns for full pattern operator reference. Add intent patterns:
  1. Training > Intents → click Patterns / +Pattern for an intent.
  2. Enter patterns; press Enter between each.
Add intent rules (trait or context tag conditions):
  1. Training > Intents → click Rules / +Rule for an intent.
  2. Enter conditions using AND/OR operators.
Add entity patterns:
  1. Training > Entities → edit icon for an entity → Patterns tab.
  2. Enter patterns.
Negative patterns (prevent incorrect intent matches):
  1. Enable: NLU Config > Ranking and Resolver Engine > Negative Patterns.
  2. Per intent: Training > Intents → select intent → Negative Patterns tab.

​
NLU Config — FM Engine Tuning

Go to Natural Language > NLU Config > Engine Tuning > Fundamental Meaning.
SettingDescription
Intent Detection using Task Name WordsDisabled: only match if input starts/ends with exact task name. Useful for tasks sharing words (e.g., multiple “order” intents). Enabled by default.
FM ThresholdMax % above the top-scoring intent for other intents to qualify. Range: 0–20% (default: 2%). Supported for English, Spanish, French, German.