Unit - II

What is Syntactic Processing?

Syntactic processing is the process of analyzing the grammatical structure of a sentence to

understand its meaning. This involves identifying the different parts of speech in a sentence,
such as nouns, verbs, adjectives, and adverbs, and how they relate to each other in order to
give proper meaning to the sentence. Let’s start with an example to understand Syntactic
Processing:

 New York is the capital of the United States of America.

 Is the United States of America the of New York capital.

If we observe closely, both sentences have the same set of words, but only the first one is
grammatically correct and which have proper meaning. If we approach both sentences with
lexical processing techniques, we can’t tell the difference between the two sentences. Here,
comes the role of syntactic processing techniques which can help to understand the
relationship between individual words in the sentence.

Difference between Lexical Processing and Syntactic Processing

Lexical processing aims at data cleaning and feature extraction, by using techniques such
as lemmatization, removing stopwords, correcting misspelled words, etc. However,
in syntactic processing, our aim is to understand the roles played by each of the words in the
sentence, and the relationship among words and to parse the grammatical structure
of sentences to understand the proper meaning of the sentence.

How Does Syntactic Processing Work?

To understand the working of syntactic processing, lets again start with an example. For
example, consider the sentence “The cat sat on the mat.” Syntactic processing would involve
identifying important components in the sentence such as “cat” as a noun, “sat” as a verb,
“on” as a preposition, and “mat” as a noun. It would also involve understanding that “cat” is
the subject of the sentence and “mat” is the object.

Syntactic processing involves a series of steps, including tokenization, part-of-speech tagging,

parsing, and semantic analysis. Tokenization is the process of breaking up a sentence into
individual words or tokens. Part-of-speech (PoS) tagging involves identifying the part of
speech of each token. Parsing is the process of analyzing the grammatical structure of a
sentence, including identifying the subject, verb, and object. The semantic analysis involves
understanding the meaning of the sentence in context.

There are several different techniques used in syntactic processing, including rule-based
methods, statistical methods, and machine learning algorithms. Each technique has its own
strengths and weaknesses, and the choice of technique depends on the specific task and the
available data.

Why is Syntactic Processing Important in NLP?

Syntactic processing is a crucial component of many NLP tasks, including machine
translation, sentiment analysis, and question-answering. Without accurate syntactic
processing, it is difficult for computers to understand the underlying meaning of human
language.Syntactic processing also plays an important role in text generation, such as in
chatbots or automated content creation.

Syntactic Structure and its Components

Syntactic structure refers to the arrangement of words or phrases to form grammatically
correct sentences in a language. It involves several components that organize and govern the
way words come together to convey meaning. The fundamental components of syntactic
structure include:
Phrases: Phrases are groups of words functioning as a single unit within a sentence. They can
be noun phrases (NP), verb phrases (VP), prepositional phrases (PP), etc.

Words/Word Classes: Words are the basic building blocks of language. Different word
classes (parts of speech) include nouns, verbs, adjectives, adverbs, prepositions, conjunctions,
determiners, etc. Each word class has its own role and function within a sentence.

Constituents: Constituents are smaller units within a sentence that form larger structures. For
instance, in the sentence "The cat chased the mouse," "the cat" and "chased the mouse" are
constituents that make up the larger sentence.

Syntax Rules: These are the rules or principles that dictate the acceptable arrangement of
words to form grammatically correct sentences in a language. They govern how words can
combine to create phrases and sentences.

Syntax Trees/Parse Trees: These graphical representations illustrate the hierarchical

structure of a sentence. They show how different constituents (phrases) are nested within one
another to form a complete sentence, with the main elements branching out into smaller units.

Syntax refers to the set of rules, principles, processes that govern the structure of sentences
in a natural language. One basic description of syntax is how different words such as
Subject, Verbs, Nouns, Noun Phrases, etc. are sequenced in a sentence.
Some of the syntactic categories of a natural language are as follows:

 Sentence(S)
 Noun Phrase(NP)
 Determiner(Det)
 Verb Phrase(VP)
 Prepositional Phrase(PP)
 Verb(V)
 Noun(N)
SyntaxTree:
A Syntax tree or a parse tree is a tree representation of different syntactic categories of a
sentence. It helps us to understand the syntactical structure of a sentence.
Example:
The syntax tree for the sentence given below is as follows:
I drive a car to my college.

Clauses: Clauses are units that contain a subject and a predicate. They can be independent
(complete sentences) or dependent (incomplete sentences that rely on an independent clause
to make complete sense).

Understanding the syntactic structure of a language helps in analyzing sentences, identifying

grammatical patterns, and constructing sentences that convey intended meanings effectively.
By understanding the grammatical structure of a sentence, computers can generate more
natural and fluent textual content.
What is Grammar?
Grammar is defined as the rules for forming well-structured sentences. While
describing the syntactic structure of well-formed programs, Grammar plays a very essential
and important role. In simple words, Grammar denotes syntactical rules that are used for
conversation in natural languages. The theory of formal languages is not only applicable here
but is also applicable in the fields of Computer Science mainly in programming languages and
data structures.
According to Chomsky's grammatical hierarchy, encompasses different types of
grammars, but the most relevant type for natural language processing (NLP) is the Context-
Free Grammar (CFG), which falls under Type 2 in Chomsky's classification.

Context-Free Grammars (CFGs) have significant relevance in NLP for modeling the syntax or
structure of natural languages. Here's how CFGs relate to natural language:

1. Syntax Modeling: CFGs are used to describe the syntax of natural languages by
defining rules that specify how valid sentences can be formed from constituents like
nouns, verbs, adjectives, etc. These grammars help capture the hierarchical structure of
sentences in a language.
2. Phrase Structure: CFGs define the phrase structure of sentences, breaking them
down into constituents such as noun phrases (NP), verb phrases (VP), prepositional
phrases (PP), etc. These constituents are formed by recursive rules defined in the
grammar.
3. Parsing: CFGs are crucial in parsing natural language sentences. Parsing involves
analyzing the syntactic structure of sentences according to the rules specified in the
grammar. Techniques like top-down and bottom-up parsing algorithms use CFGs to
generate parse trees for sentences.
 4. Formal Representation: CFGs formalize the rules governing the arrangement of
words in a sentence. These rules dictate how words and phrases can be combined to
form grammatically correct sentences.

Example to generate parse tree using grammmers:

Sentence: The man ate the burger
Parse Tree:

The Sentence-to-sentence symbol is called as Top-down Parsing.

The above rule is called as production rule. In natural language processing (NLP), a
production rule, also known as a rewrite rule or a grammar rule, describes how symbols in a
formal grammar can be replaced by other symbols or sequences of symbols. These rules
define the structure and syntax of a language, providing guidelines for generating valid
sentences or phrases.

Formally, a production rule consists of two parts:

 1. Left-hand side (LHS): This is the symbol or non-terminal on the left side of the rule.
It represents a syntactic category or a symbol that can be expanded or replaced
according to the rule.
2. Right-hand side (RHS): This is the sequence of symbols on the right side of the rule.
It consists of terminals (actual words or symbols representing the basic units of the
language) and/or non-terminals (symbols that can be further expanded by other
production rules).

Bottom Up parsing:
The essence of bottom-up parsing lies in starting with individual words or tokens and
gradually constructing larger syntactic units by applying grammar rules until the entire
input is successfully parsed.

Fig: Bottom up parsing

It involves the process of analyzing and constructing the structure of sentences or phrases by
starting from the individual words or tokens and working upwards to form higher-level
constituents based on the rules defined in a given grammar.

This approach is often used with context-free grammars (CFGs) or other formal grammars
and employs parsing techniques that iteratively build constituents from the bottom (individual
words) to the top (complete sentence or phrase).

Mathematically, a grammar G can be written as a 4-tuple (N, T, S, P) where,

N or VN = set of non-terminal symbols, or variables.
T or ∑ = set of terminal symbols.
S = Start symbol where S ∈ N
P = Production rules for Terminals as well as Non-terminals.
It has the form α → β, where α and β are strings on VN ∪ ∑ and at least one symbol of α
belongs to VN.

Toward efficient parsing

Efficient parsing in Natural Language Processing (NLP) is crucial for various language
understanding tasks. Here are ways to achieve efficiency in parsing within NLP

Optimized Algorithms: Employ parsing algorithms tailored for NLP tasks. Techniques like
transition-based parsing (e.g., Shift-Reduce parsers) or chart-based parsing (e.g., CYK) can
efficiently parse sentences.

Dependency Parsing: Utilize dependency parsers that focus on relationships between words
rather than phrase structure. Dependency parsing often leads to faster and simpler parsing.

Neural Network Models: Leverage neural network architectures for parsing, such as graph-
based parsers or transformer models (e.g., BERT, GPT) that excel in handling contextual
information and have shown efficiency in various parsing tasks.

Incremental Parsing Models: Use models that allow for incremental parsing, enabling real-
time analysis and faster processing of incoming language input.

Domain-Specific Parsers: Develop parsers specifically tailored for certain domains or types
of text. These parsers can focus on the specific linguistic patterns prevalent in those domains,
leading to faster and more accurate parsing.

Parallel Processing: Employ parallel computing techniques to process multiple sentences

concurrently, speeding up parsing, especially in large-scale NLP tasks.
Feature Engineering and Selection: Optimize feature sets used in parsing models to reduce
computational overhead. Feature selection and dimensionality reduction techniques can
streamline parsing without sacrificing accuracy.

Language-Specific Optimizations: Implement language-specific optimizations that leverage

the characteristics and structures inherent in certain languages. This includes language-
specific rules or techniques that can expedite parsing.

Incremental Model Updates: For applications where the parsing model needs to adapt to
new data continuously, incremental learning techniques can be employed to update the model
efficiently without retraining from scratch.

Hybrid Approaches: Combine different parsing techniques or models to take advantage of

the strengths of each, creating hybrid systems that are both efficient and accurate.

Efficient parsing in NLP is essential for tasks like information extraction, sentiment analysis,
machine translation, and question answering. Balancing accuracy and speed is often a
challenge, and the choice of parsing method depends on the specific requirements of the NLP
application.