Introduction to Grammar in NLP structing sentences in a Tanguage used to understand and srammar in NLP is a set of rules for cons analyze the structure of sentences In text data. This includes identifying parts of speech Such as nouns, verbs, and adj Subject and predicate of a sentence, and ‘identifying the relationships between Words and phrases, So humans, 576 tale ca as eae Gy understandable to other humans ‘and not foes To make computers understand language, they must have a structee to follow. Syntax describes explicit. What is Grammar? Grammar is defined as the rules for f sroalal role in describing the syntactic sueture of well-formed programs, like denoting the 'als a5 well as Non-terminals, © Ithas the form 0 O03, where aand B are strings on Oeuy VNUY, and at VN least one symbol of a belongs to Syntax also refers to the “ay Words are arranged together, Let us see some basic ideas related to * Constituency: Groups of words ‘may behave as a single unit or phrase ~A constituent, for example, lke @ Noun phrase, * Grammatical relations: These are the formalization of ideas from \raditional grammar. Examples irclude- subjects and objects, hese are the relations between words and phrases, for example, a Verb followed by an infinitive verb, * Regular languages and part of sheech: Refers to the way words are array nged together but gaunt Support easily, Examples ore Constituency, Grammatical re ti ond opener al ‘lations, and Subcategorization ei* Syntactic categories and their common denotations in NLP: np - noun phrase, vp - verb phrase, inteatence, det - determiner (article), n = noun, tv - transitive verb (takes an object), iv - intransitive verb, prep - preposition, pp - prepositional phrase, adj - adjective Types of Grammar in NLP Let us move on to discuss the types of grammar in NLP. We will cover three types of grammar: context-free, constituency, and dependency. Context Free Grammar Context-free grammar consists ofa set of rules expressing how symbols of the language can be Brouped and ordered together and a lexicon of words and symbols, * One example rule isto express an NP (or noun phrase) that can be composed of either a * Context-ree rules can also be hierarchically embedded, so we can combine the previous rules with others, like the following, that express facts about the lexicon: Det > a Det -> the Noun -> e@ flight * Formalism in rules for context-free grammar: A sentence in the language defined by a CFG is a Series of words that can be derived by systematically applying the rules, beginning with a rule that has s on its left-hand side. © Use of parse tree in context-free grammar: A convenient way to describe a parse is to Show its parse tree, simply a graphical display of the parse. © Aparse of the sentence is a series of rule applications in which a syntactic category is @ replaced by the right-hand side of a rule that has that Category on its left-hand side, and the final rule application yields the sentence itself,” > de > the n vp =? «Example: A parse of the sentence "the giraffe dreams” s:$ > np vp => det n vp =? the n VP the giraffe vp => the giraffe iv => the giraffe dreams hy, AreamR sJenb ee np J det ne ~~ P VP ty ne Ve \- \ iN iv det 7 He ee : = one | n—> gqivaffe The gqrefte dreamr ju eat, > dead «fwe look at the example parse tree for the sample sentence in the illustration the giraffe dreams, the graphical illustration shows the parse tree for the sentence s Wecan see that the root of every subtree has @ ‘grammatical category that appears on the left- hand side ofa rule, and the children of that root are identical to the elements on the right-hand side of that rule. Classification of Symbols in CFG The symbols used in Context-free grammar are divided into two classes. «The symbols that correspond to words in the language, for example, the nightclub, are called terminal symbols, and the lexicon is the set of rules that introduce these terminal symbols. 4s The symbols that express abstractions over these terminals are called non-terminals. ineach context-free rule, the item to the right of the arrow (>) isan ordered list of one or more terminals and non-terminals, and to the left of the arrow is 3 single non-terminal symbol expressing some cluster or generalization. - The non-terminal associated with each word in the lexicon is its lexical category or part of speech. Context Free Grammar consists ofa finite set of grammar rules that have four components: 8 Set of Non-Terminals, a Set of Terminals, a Set ‘of Productions, and a Start Symbol. FG ean also be seen as a notation used for describing the languages, a superset of Regular grammar. CFG consists ofa finite set of grammar rules having the following four components «Set of Non-terminals: itis represented by V. The non-terminals are syntactic variables that denote the sets of strings, which help define the language generated with the help of grammar. «Set of Terminals: It is also known as tokens and is rey resented by 5. Strit med help of the basic symbols of terminals. i ere hd = Set of Productions: Itis represented by P. The Stel bomen by P. The set explains how the terminals and non-terminals,Every production consists of the following components: + Non-terminals are also called variables or placeholders as they stand for other symbols, either terminals or non-terminals. They are symbols representing the structure of the language being described. Non-terminals are a set of production rules specifying how to replace a non-termina) symbol with a string of symbols, which can include terminals, words or characters, and other non-terminals. * _ Start Symbol: The formal language defined by a CFG is the set of strings derivable from the designated start symbol. Each grammar must have one designated start symbol, which is often called s. * Since context-free grammar is often used to define sentences, S is usually interpreted as the sentence node, and the set of strings that are derivable from S is the set of sentences in some simplified version of English. Issues with using context-free grammar in NLP: oui ted expressiveness: Context-free grammar isa limited formalism that cannot capture certain linguistic phenomena such as idiomatic expressions, coordination and ellipsis, and even long-distance dependencies. * Handling idiomatic expressions: CFG may also have a hard time handling idiomatic expressions or idioms, phrases whose meaning cannot be inferred from the meanings of the idual words that make up the phrase. * Handling coordination: CFG needs help to handle coor clauses with a conjunction, * Handling ellipsis: Context-free of one or more words from a s ion, which is linking phrases or Grammar may need help to handle ellipsis, which is the omission entence that is recoverable from the context. The limitations of context-free dependency grammar which is powerful but more complex to implement * The constituents can be any word, group of words or phrases in Constituency Grammar. The 808 of constituency grammar is to organize any sentence into its constituents using their Properties. * Characteristic properties of constituency grammar and constituency relation: © Allthe related frameworks view the sentence structure in terms of constituency relation, © To derive the constituency relation, we take the help of subject-predicate division of Latin as well as Greek grammar. in constituency grammar, we study the clause structure in terms of noun phrase NP and verb phrase vp. * The properties are derivedie i NP Ne \ gaa ees be ii dey Chane Deb hou The. Cat Look at a sample parse tree: Example sentence - "The dog chased the cat.” in this parse tree, the sentence is represented by the root node S (for sentence). The sentence is divided into two main constituents: NP (noun phrase) and VP (verb phrase). ‘The NP is further broken down into Det (determiner) and Noun, and the VP is further broken down into V (verb) and NP. Each of these constituents can be further broken down into smaller constituents. Constituency grammar is better equipped to handle context-free and dependency grammar limitations. Let us look at them: Constituency grammar is not language-specific, making it easy to use the same model for ‘multiple languages or switch between languages, hence handling the muttlingual issue plaguing the other two types of grammar. Since constituency grammar uses a parse tree to represent the hierarchical relationship between the constituents of a sentence, it can be easily understood by humans and is more intuitive than other representation grammars. + Constituency grammar also is simple and easier to implement than other formalisms, such as dependency grammar, making it more accessible for researchers and practitioners. * Constituency grammar is robust to errors and can handle noisy or incomplete data. * Constituency grammar is also better equipped to handle coordination which is the linking of phrases or clauses with a conjunction.ced on the dependency Dependency Grammar Dependency Gramma the opposite of constituentY & mmar ad is based on is te to the constituency grammar because it 1 ks phrasal and dependency relation relation. It is oppos! Let us look at some fundamental points about Dependency &F re dependent UPO! amar. The ver 5 states that words of a semtence ¢ in dependency 62! 1s Dependency Grammar f sentence. These Words 27° onnected by directed links | sermdered the coneer oF te UE structure. , «s Dependency Grammar organlcee words ofa sentence according to their dependencies Every other syntactic units Tonnected to the verbin terms ccreediected link, THese SYA nits are called dependencies é, Sane ofthe words in a sentence behaves as a root, and all the other words except tha word itself are linked direct °F rerreetly wth the root using eee dependencies 5 These dependencies represen r ationships among the words 1 sentence, and dependency grammar is used (° infer the structure and semantic dependencies ? between the words. Dependeney grammar sufers fom some limitations; let us understand them further: ‘ambiguity when it comes tO interpreting the hen dealing with grammar has issues with ‘ds, which are particularly challenging w! 1s between Wor ‘word order variations. flections or complex fiso requires labeled data to tra ‘ambiguity: Dependency grammatical relationshi fanguages that have rich in «ats annotation: Dependency Parsing consuming and difficult to obtain Handling long-distance dependencies: Dependency parsing al ance endencis in some coses where relationships betwe very far apart, making ilficltfO ‘accurately capture the gram sentence. + Handling elli phenomena and coordin. in the model, whic! Iso has issues with handling !one- ‘en words ina sentence may De ratical structure of the ‘on: Dependency grammar also has 3 hard time handling such as ellipsis od by the direct relationships between words, jptured by constituency grammar- “The limitations of dependency grammar Cot be mitigated by using constituency grammet which, ough less powerful, but more intuitive wind easier to implement. We can also use & hybrid approach where both constituency nnd dependency are used together, and it can be beneficial. and coordinati that are not capture’ ration, which are typically caf «comparing Constituency grammar with Dependency grammar: Constituency gramme focuses on grouping words into phrases and clauses, while dependency grammar focuses on the reais petween individual words. Each word in a sentence © represented by anode in a Tependency graph, and the edges between nodes re resent the i i deeded aay p grammatical relationships weet ependency grammar is typically more powerful for some langu captures the relationships between the words more accurately, to implement and less intuit © Constituency grammar is mor expressive. jages and NLP tasks as it but also more complex ive. re intuitive and easier to implement but can be lessProcessing (NLP), parsing makes complex user instruction portant link between human language and Parsing is important for using NLP 6 with answering questions closely with illus of in Natural Language for computers to understand. It acts as an im the digital world, opening the door to various uses. technology in our daily interactions with computers. It help: translating, and more. Join us as we examine parsing more real-life applications to enhance your understanding Parsing is the key to a world of NLP applications, where computers read, write, and interpret text ike never before. In this blog, we will discover what parsing in NLP is along with its types and applications. But before moving ahead, ponder over a thought What drives the magic behind parsing in NLP? Parsing is the process of examining the grammatical structure and relationships inside a given sentence or text in,natural language processing (NLP). It involves analyzing the text to determine the roles of specific words, such as nouns, verbs, and adjectives, as well as their interrelationships. This analysis produces a structured representation of the text, allowing NLP computers fords in a phrase connect to one another. Parsers expose endency trees that illustrate ations of to understand how w the structure of a sentence by constructing parse trees or dep the hierarchical and syntactic relationships between words This essential NLP stage is crucial for a variety of language understanding tasks, which allow machines to extract meaning, provide coherent answers, and execute tasks such as machine translation, sentiment analysis, and information extraction If you want to know more about ‘What is Natural Language Processing?’ you can go through this Natural Language Processing Using Python course! ai rarsthginner | orsing’ ; i ] L t, Basic Parserin NLP ZYachines to perceive the quage processing of Parsing in NLP NLP, allowing m: Types THe ypes of parsing ate the €or steps in i eMsture and meaning of the fox hich is required for a variety of fai sttivties. There are two main types ‘of parsing in NLP which are as follows ical structure. It involves looking at and word Syntactic Parsing, ' ' be ic parsin. leals with a sent tence's grammati oyna sentence boundaries, Peentence to determine parts of speech, relationships. The two most common approaches included are as follow i "sonstituency Parsing: Constituency Parsing builds parse trees that break dow? Sentence into its constituents, such 3% oun phrases and verb phrases, displays a_sentence’s hierarchical structure, demonstrating how words are ranged into bigger grammatical units. Dependency Parsing: Dependency parsing depicts grammatical links betwee? words by constructing a tree structure in ‘Bich each word in the sentence dependent on another. It is frequently tod in tasks such as information extraction and machine ‘on word relationships such iranslation because it focuses as subject-verb-object relations. ntence’s meaning oF text of a certain task a variety of NLP g, and text f actionable Semantic Parsing ond syntactic structure to extract a Se Semantic parsing goes bey Serrantics. It attempts to understand the roles of words in the con! Sefhnow they interact with one another. Semantic parsing is utilized in applications, such as question answerng, knowledge base populatin: oPrerstanding. it is essential for activities requiring the extraction of information from text. Parsing Techniques in NLP. tween a sentence and its grammar is deri ed from a parse tree. The fundamental link bel A parse tree is a tree that defines how the grammar was utilized to construct the asFtence. There are mainly two parsing techniques, ‘commonly known as top-down and bottom-up. jown Parsing + Aparse tree is @ tre sentence. Using the to} tree from the root node «The procedure begins with the assumption t selected start symbol S. «Fhe next step is to find the tops of all the trees that can begin with S by looking at the grammatical rules with S on the left-hand side, which generates all the possible trees. cpicecaty parsing is a search with a specific objective in mind, +1 ekompts'to replicate the inital creation process by rederiving the sentence from tha sian symbol, and the production tree is recreated from the top down. «Top-down, left-to-right, and backtracking are prominent search strategies that are used in this method. see a rsh" begins with the root node labeled S, ie., the starting symbol xt productions with the left-hand side expands the internal nodes using the ne: equal to the internal node, and continues until leaves are part of speech (terminals). ts of speech, do not match the input string, we must go + Ifthe leaf nodes, or pat back to the most recent node processed and apply it to another production. that defines how the grammar was utilized to construct the p-down approach, the parser attempts to create @ parse ‘S$ down to the leaves. that the input can be derived from theoO sting time investigating trees ntage of never Wi es subtrees that cannot find a technique has the adva er examin’ The top-down that cannot result in S, which indicates it nev place in some rooted tree. Bottom-Up Parsing ~ Bottom-up parsing begins with the words of input and attempts to create trees from the words UP, again by applying grammar rules one at a time. » The parse is successful if it builds a tree rooted in the start symbol S that includes all of the input. Bottom-up parsing is a type of data-driven search. It attempts to reverse the manufacturing process and return the phrase to the start symbol S. + Itreverses the production to reduce the string of tokens to the beginning Symbol, and the string is recognized by generating the rightmost derivation in reverse. The goal of reaching the starting symbol S is accomplished through a series of reductions; when the right-hand side of some rule matches the substring of the input string, the substring is replaced with the left-hand side of the matched production, and the process is repeated until the starting symbol is reached. «Bottom-up parsing can be thought of as a reduction process. Bottom-up parsing is the construction of a parse tree in postorder. Considering the grammatical rules stated above and the input sentence “John is playing agame”, The bottom-up parsing operates as follows: _AntelliPoct Noun AuV —Yptb No oo is tole playing game relat ali nln AY Veo Non shin Is playing gameLe Let's consider the grammar rules: Sentence = S = Noun Phrase (NP) + Verb Phrase (VP) + Preposition Phrase (PP) Take the sentence: "John is playing a game”, and apply Top-down parsing Part of the speech erp me not match the peut string, backtrack to the node S, since PNoun is ‘matched.Conclusion Grammar is defined as the rules for forming well-structured sentences and plays an essential rae in denoting the syntactical rules that are used for ‘conversation in natural languages: Syntax the branch of grammar dealing with the structure and formation of sentences ina language governing how words are arranged to form phrases, clauses, and sentences. Context-free grammar is a formal grammar uses using a set of production rules describing how to rep symbols called terminals and non-terminals. constituency grammar is another type of formal grammar to analyze the structure of sentences ina language using the Idea that sentences can be broken down into smaller units called constituents with a specific grammatical function. Dependency grammar is formal grammar to analyze the grammatical relationships in nlp using dependencies between words to form the structure of the sentence. 1d to recognize the set of strings in a language lace non-terminals with another string ofParsers and Its Types In NLP As previously stated, a parser is essentially a procedural interpretation of grammar. It ‘searches through the space of a variety of trees to find the best tree for the provided sea et's have a look at some of the accessible parsers below + Recursive Descent Parser SA top-down parser that iteratively breaks down the highest-level grammar ri tibrules is known as a recursive descent parser. It is frequently implemented as a set of recursive functions, each of which handles a certain grammatical rule. «= ‘This style of parser is frequently employed in hand-crafted parsers for simple programming languages and domain-specific languages. + Shift-Reduce Parser Hee ghiftreduce parser is a sort of bottom-up parser that starts with the input Ait builds a parse tree by performing a series of shift (transfer data to the stack) and reduction (apply grammar rules) operations: Shiftreduce parsers are used in programming language parsing and are frequently used with LR (Lefto-right, Rightmost derivation) or LALR (Look-Ahead LR) parsing techniques + Chart Parser © A chart parser is a sort of parsing algorithm that efficiently parses words by using dynamic programming and chart data structures. To reduce unnecessary work, it stores and reuses intermediate parsing results Early parser is a type of chart parser that is commonly utilized for parsing context-free grammars. + Regexp Parser 3. A regexp (regular expression) parser is used to match patterns and extract text. It scans a larger text or document for substrings that match a specific regular expression pattern. © Text processing and information retrieval tasks make extensive use of regexp parsers. Each of these parsers serves a different purpose and has its own set of benefits and drawbacks. The parser chosen is determined by the nature of the parsing task, the grammar of the language being processed, and the application's efficiency requirements. How Does the Parser Work? The first step is to identify the sentence's subject. The parser divides the text sequence into a group of words that are associated with a phrase. So, this collection of words that are related to one another is referred to as the subject. Syntactic parsing and parts of speech are context-free grammar structures that are a eutig Eine erenvement of words. It is not determined by the context. i ; \ “ pe mer inpatert altoyemerbe, 8 that grammar is always syntactically valid,Applications of Parsing in NLP Parsing is a key natural language processing approach for comprehending the grammatical structure of natural language text. Parsing Is important in NLP for various reasons. Some of them are mentioned below: analyzing and jetermining the syntactic structure of sentences Syntactic Analysis: Parsing helps in d en words, by detecting parts of speech, phrases, and grammatical relationships betwe This information is critical for understanding sentence grammar. Named Entity Recognition (NER): NER parsers can detect and classify entities in text, such as people's, organizations, and locations’ names, among other things. This is essential for information extraction and text comprehension. Semantic Role Labeling (SRL): SRL parsers determine the semantic roles of words in a sentence, such as who is the “agent,” “patient,” or ‘instrument’ in a given activity. It is essential for understanding the meaning of sentences. Machine Translation: Parsing can be used to assess source language syntax and generate syntactically correct translations in the target language. This is necessary for machine translation systems such as Google Translate. Question Answering: Parsing is used in question-answering systems to help break down a question into its grammatical components, allowing the system to search a corpus for relevant replies. Text Summarization: Parsing is the process of extracting the essential syntactic and semantic structures of a text, which is necessary for producing short and coherent summaries. Information Extraction: Parsing is used to extract structured information from unstructured text, such as data from resumes, news articles, or product reviews.Transition Network Grammar * Transition network grammar is another useful formalism * It consists of nodes and labeled arcs : initial or start state is the first node of the network start at the initial state, and traverse an arc if the current word in the sentence is in the category on arc -- if the arc is followed, the current word is updated to the next word -- a phrase is a legal one if there is path from the initial state to a pop arc Transition networks (TN) are made up of a set of finite automata and represented within a graph system. The edges indicate transitions and the nodes the states of the single automata. Each automaton stands for a non-terminal symbol and is represented by its own network. The edges of each single network are denoted by non-terminal or terminal symbols and thus refer to other networks or final states. If the structure of a transition network also allows for recursive processes, for example, in the substitution of an object by another object belonging to a higher hierarchy level (e.g. a verb becomes a verbal phrase), this type of network is known as a recursive transition network. A path traversing the transition network starts at a first network and, beginning at the starting node, passes along the single edges. When it encounters a non-terminal symbol, the system branches like a sub- program to the corresponding network until finally all non-terminal symbols have been substituted. If different substitution possibilities are available, several paths between starting state and final state of the respective finite automaton exist. Figure 5.1 shows a transition network for expressions in natural language which may generate expressions such as “conductor likes singer,” “a singer hates the conductor,” “a singer likes a conductor hates the singer”.nominal phrase «© 3 @ : @ fa © tes @ a enna please — the ~—= © OS «6 @ ath A transition network for natural language expressions Augmented transition network ATNis a type of graph theoretic structure used in the operational definition of formal languages, used especially in parsing relatively complex natural_lanquages, and having wide application in artificial intelligence. An ATN can, theoretically, analyze the structure of any sentence, however complicated. ATN are modified transition networks and an extension of RTNs!*=0= ATNs build on the idea of using finite state machines jodel) to parse sentences. W. A. Woods in "Transition Network Grammars for Ni anguage Analysis" claims that by adding @ recursive mechanism to a finite state model, parsing can be achieved much more efficiently. Instead of building an automaton for a particular sentence, a collection of transition graphs are built A grammatically correct sentence is parsed by reaching a final state in any state graph. Transitions between these graphs are simply subroutine calls from one state to any initial state on any graph in the network. A sentence is determined to be grammatically correct if a final state is reached by the last word in the sentence. This mode! meets many of the goals set forth by the nature of language in that it captures the fegularities of the language. That is, if there is a process that operates in a number of environments, the grammar should encapsulate the process in a single structure. Such encapsulation not only simplifies the grammar, but has the added bonus of efficiency of operation. Another advantage of such a model is the ability to postpone decisions. Many grammars use guessing when an ambiguity comes up. This means that not enough is yet known about the sentence. By the use of recursion, ATNs solve this inefficiency by postponing decisions until more is known about a sentence. Recursive transition network ('RTN') is a graph theoretical schematic used to represent the rules of a context-free grammar. RTNs have application to programming languages, natural_language and lexical analysis Any sentence that is constructed according to the rules of an RTN@is said to be ‘well-ormed”. The structural elements of a well-formed sentence may also be well-formed sentences by themselves, or they may be simpler structures. This is why RTNs are described as recursive."Figure 5.1 shows a transition network for expressions in natural language which may generate expressions such as “conductor likes singer,” “a singer hates the conductor,” “a singer likes a conductor hates the singer” cual phrase sector A transition network for natural language expressions Augmented transition network ATNis a type of graph theoretic structure used in the operational definition of formal_tai used especially in parsing relatively complex natural languages, and having wide application (p attficialintellence. An ATN can, theoretically, analyze the structure of any sentence however complicated. ATN are modified transition networks and an extension of RTNS======2 ATNs build on the idea of using finite state machines (Markov model) to parse sentences. W. A Woods in “Transition Network Grammars for Natural Language Analysis" claims that by adding Inulcad oremechanism to a finite state model, parsing can be achieved much more efficiently |nstead of building an automaton for a particular sentence, a collection of transition graphs are bare A. grammatically correct sentence is parsed by reaching a final state in any state graph, Trarcltens between these graphs are simply subroutine calls from on le state to any initial state on any graph in the network. A sentence is determined to be grammatically correct if a final state is reactrea by the last word in the sentence. This model meets many of the goals set forth by the nature of language in that it captures the ‘equiarities of the language. That i, if there is a process that operates in a number of enviconmente the grammar should encapsulate the process in a sin simplifies the grammar, but has the added bonus of such a model is the ability to postpone decisions. Many grammars use an ambiguity comes up. This means that not enough is yet known about the senten recursion, ATNs solve this ir sentence. 1ce. By the use of inefficiency by postponing decisions until more is known about 2 Recursive transition network Tre 8,2 stat theoretical schematic used to represent the rules of a contextfree grammar. RINs have application to programming languages, natural language and lexical analysis. Any sentence that is constructed according to the rules of an RTN@ is said to be “well-formed”, The Structural elements of a well-formed sentence may also be well-formed sentences by themselves, or they may be simpler structures. This is why RTNs are described as recursive.”Top- Down Chart Parsing Earley parsers are among the most general parsers out there, and they can parse any context-free language without restriction. Earley parsing algorithm is an efficient top-down parsing algorithm anarveids some of the inefficiency associated with purely naive search with the same top-down strategy, and they are reasonably fast on most practical grammars and are easy to implemen" Earley parsing uses a dynamic programming ‘mechanism to avoid recomputation to solve exponential time complexity problems. We will cove, the top-down parsing in brief and then Sree the working ofthe Earley algorithm in detail in this article of parsing algorithms - top-down parsing and bottom. up suffer from common limitations like left recursion, left tial time to execute. Earley parser deals re we use a memoization Introduction We have seen majorly two categories parsing. Both of the parsing methods st factoring, efc., and their implementation takes expones with these problems using a dynamic programming ‘mechanism whei technique ‘Top-down Parsing In Top-down parsing, we build the parse tres opposed to bottom-up parsing, where we build root(top). In top-down parsing, we generate the parse the grammar and use the leftmost derivation to build its brane parsed using the following steps: by starting from the root and going to the leaf ‘the leaves first and then move upwards toward the tree from the root as the starting symbol of hhes. A string in top-down parsing is mar with a start symbol is applied. ‘To derive the input string, first, a production in gram le of a non-terminal must be applied to derive the ‘At each step, the top-down parser identifies which rul input string. ‘As the next step, Example: Consider a grammar $->aAdS->aAd, A->blbcA—>blbe, and the input string that we want to parse is abd. the terminals in the production are matched with the terminals ofthe input string The top-down parser will start creating a parse tree with the starting symbol S.Oo Now the first input symbol ‘a! matches the first leaf node of the tree. So the pafser will move head and find a match for the second input symbol '." But the next leaf node of the parse tree aetontemminal which has two production rules, so the parser identifies a production rule A~>bA->b pe Now the next leaf node matches with the last character of the input string. Hence the parsing is ‘completed. Pros and Cons of top-down Parsing ‘Advantages of top-down Parsing ‘The top-down parsing is very simple It is very easy to identify the action decision {the grammar rule to pick up next) in a top-down parser Disadvantages of top-down Parsing ‘Top-down parsing is unable to handle left recursion in the present grammar. ‘Some recursive descent parsing may require backtracking In top-down parsing, there can be multiple In top-down parsing, there can be many possible acceptable partial parses of the first (left-most) components of the string that are repeatedly regenerated The Early Algorithm This revisiting of early partial solutions within the full parse structure is the result of later backtracking requirements of the search and can become exponentially expensive in large parses. Dynamic programming provides an efficient alternative where partial parses, once generated, are saved for reuse in the overall final parse of a string of words. Memoization And Dotted Pairs In parsing with Earley’ algorithm, the memoization of partial solutions (partial parses) is done with a data structure called a chart. This is why the various alternative forms of the Earley approach to parsing are sometimes called chart parsing, The chart is generated through the use of dotted grammar rules.Transition Network Grammar > Transition network grammar is another useful formalism. «It consists of nodes and labeled arcs : -_ initial or start state is the first node of the network o ftart at the initial state, and traverse an arc if the current word in the sentence is in the category on arc -- if the arc is followed, the cl --a phrase is a legal one if there is path from the initial state t CpG" urrent word is updated to the next word ‘0 a pop arc os) get x) fs ee Transition networks (TN) are made up of a set of finite automata and represented within a graph system, The edges indicate transitions and the nodes the states of the single automata. Each automaton stands for a non-terminal symbol and is represented by its own network. The edges of each single network are denoted by non-terminal or terminal symbols and thus refer to other networks or final states. If the structure of a transition network also allows for recursive processes, for example, in the substitution of an object by another object belonging to a higher hierarchy level (e.g. a verb becomes a verbal phrase), this type of network is known as a recursive transition network. A path traversing the transition network starts at a first network and, beginning at the starting node, passes along the single edges. When it encounters a non-terminal symbol, the system branches like a sub- program to the corresponding network until finally all non-terminal symbols have been substituted. If different substitution possibilities are available, several paths between starting state and final state of the respective finite automaton exist. 1il Grammar For Natural Languages Meet eu eckenm sree Grammar in NLP is a set of rules for constructing sentences in a language used to understand and analyze the structure of sentences in text data, This includes identifying parts of speech such as nouns, verbs, and adjectives, determining the Subject and predicate of a sentence, and identifying the relationships between words and phrases. As humans, we talk in a language that is easily understandable to other humans and not computers. To make computers understand language, they must have a structure to follow. Syntax describes a language's regularity and productivity, making sentences’ structure explicit. The word syntax here refers to the way the words are arranged together. Regular languages and parts of speech refer to how words are arranged together but cannot support easily, such as grammatical or dependency relations. These can be modeled by gramnmar, Let us dive deép into Grammar and explore the terms discussed here What is Grammar? Grammar is defined as the rules for forming well-structured sentences, Grammar also plays an essential role in describing the syntactic structure of well-formed programs, like denoting the syntactical rules used for conversation in natural languages. + In the theory of formal languages, grammar is also applicable in Computer Science, mainly in programming languages and data structures. Example - In the C programming language, the precise grammar rules state how functions are made with the help of lists and statements. + Mathematically, a grammar G can be written as a 4-tuple (N, T, S, P) whe: © Nor VN=set of non-terminal symbols ot variables, T or 5 = set of terminal symbols. S = Start symbol where $ € N P = Production rules for Terminals as well as Non-terminals. Ithas the form @—-Ba—, whiere a and f are strings on VNUY;VNUY,, and at least one symbol of a belongs to VN 2 000 . Syntax Each natural language has an underlying structure usually referred to under Syntax. The fundamental idea of syntax is that words group together to form the constituents like groups of words or phrases which behave as a single unit. These constituents can combine to form bigger constituents and, eventually, sentences,+ Syntax describes the regu ity and productivity of a language making explicit the smeaure of sentences, and the goal of syntactic analysis or parsing is to detect ia Sentence is correct and provide a syntactic structure of a sentence Syntax also refers to the way words are arranged together. Let us see some ba sic ideas related to syntax: * Constituency: Groups of words may behave as a single unit or Phrase - A constituent, for example, like a Noun phrase Srammatical relations: These are the formalization of ideas from traditional ¢ Examples include - subjects and objects * Subcafegorization and dependency relations: These are the relations between words and phrases, for example, a Verb followed by an infinitive verb. + Regular languages and part of speech: Refers to the way words are arranged tog but cannot support easily. Examples are Constituency, Grammatical relations, at Subcategorization and dependency relations yrNactic categories and their common denotations in NLP: np - noun phrase, vp verb phrase, s- sentence, det - determiner (article), n- noun, tv - transitive verb (ales an object) iv - intransitive verb, prep - preposition, pp - prepositional phrase, ad - adjective ammar, ary Verb (abbreviated Aux) is a verb that adds functional or grammatical meaning to ihe clause in which it occurs, so as to express tense, aspect, modality, voice. emiphas s ove Auxiliary verbs usually accompany an infinitive verb or a participle, which respectively provide the raat eamantic content of the clause."! An example is the verb have in the sentence / have finished my lunch, Here, the auxiliary have helps to express the perfect aspect along with the Participle, finished. Some sentences contain a chain of two of more auxiliary veros Auxiliary verbs ate also called helping verbs, helper verbs, or (verbal) auxiliaries. Research has been con ed into split inflection in auxiliary verbs, <. fraits across languages Auriiary verbs typically help express grammatical tense, aspect, mood, and voice. They generally spear together with an infinitive. The auxiliary is said o "help" the infinitive. The auxilary verbs oa language form a closed class, i.e, there is a fixed, relatively small number of them Widely acknowledged verbs that can serve as auxiliaries in English and many related <™ languages are the equivalents of be to express passive voice, and have (and sometimes be) to express perfect aspect or past time reference.” as In some treatments, the copula be is classed verb, eg, ‘The bird is in the tree. ~ is serves «verb, Derattons of auxiliary verbs are not always consistent across languages, or even among authors discussing the same language. Modal verbs may or may not be classified as auxliaries, jebending on the language. In the case of English, verbs are often identified as auxiliaries based on their grammatical behavior, as described below. In some cases, verbs that function similarly to auxiliaries, but are not considered full members of that class (perhaps because they Carty some independent lexical information), are called semi-auxiliaries. In French, for example, verbs such as devoir (have to), pouvoir (be able to), aller (be going : in auxiliary even though it does not "help" another ‘@ copula with a predicative expression not containing any otherto), vouloir (want), faire (make), and laissor (lel), when used together with the infinitive of another verb, can be called semi-auxiliaries.""" There has also been a study on auxiliary verb constructions in Dravidian languages. AA list of verbs that (can) function as auxiliaries in English is as follows: be, can, could, dare, do, have, may, might, must, need, ought, shall, should, will, would The status of dare (nol), need (nol), and ought (to) is debatable! and the use of these verbs as auxiliaries can vary across dialects of English. If the negative forms can't, don't, won't etc. are viewed as separate verbs (and not as contractions), then the number of auxiliaries increases. The verbs do and have can also function as full verbs or as light verbs, which can be a source of confusion about their status. The modal verbs (can, could, may, might, must, shall, should, will, would, and dare, need and ought when included) form a subclass of auxiliary verbs. Modal verbs are defective insofar as they.cannot be inflected, nor do they appear as gerunds, jnfinitives, or participles. The following table summarizes the auxiliary verbs in standard English and the meaning contribution to the clauses in which they appear. Many auxiliary verbs are listed more than once in the table based upon discernible differences in use. Light verbs ‘Some syntacticians distinguish between auxiliary verbs and light verbs “2 The two are similar insofar as both verb types contribute mainly just functional information to the clauses in which they appear. Hence both do not qualify as separate predicates, but rather they form part of a predicate with another expression — usually with a full verb in the case of auxiliary verbs and usually with a oun in the case of light verbs. In English, light verbs differ from auxiliary verbs in that they cannot undergo inversion and they cannot take not as a postdependent. The verbs have and do can function as auxiliary verbs or as light verbs (or as full verbs). When they are light verbs, they fall the inversion and negation diagnostics for auxiliaries, e.g, ; Note that in some dialects (for example, the West and South West dialects of Hiberno-English), the inversion test may sound correct to native speakers. a, They had a long meeting. b. "Had they a long meeting? - Light verb had fas the inversion test. . “They had not a long meeting, - Light verb had falls the negation test. a. She-did a report on pandering politicians. b. *Did she a report on pandering politicians? - Light verb did fais the inversion test ¢. *She did not a report on pandering politicians. - Light verb did fas the negation test (In some cases, though, have may undergo auxiliary-type inversion and negation even when itis not used as an auxiliary verb - see t-auxiliary inversion § Inversion with other types of verb.) ‘Sometimes the distinction between auxiliary verbs and light verbs is overlooked or confused. Certain verbs (e.g., used fo, have fo, etc.) may be judged as light verbs by some authors, but as auxiliaries by others.Verb Phrase Verb phrases generally are divided among two types: finite, of which the head of the phrase is 8 finite verb; and nonfinite, where the head is a nonfinite verb, such ag pn infiive, participle or gerund. Phrase structure arammars acknowledge both types, Put dependency grammars treat the subject as just another verbal deperdent, ang they do not Kronze jhe finite verbal phrase constituent. Understanding verb phrase analysis deronds on ig which theory applies in context In phrase structure grammars In phrase structure grammars such as generative grammar, the verb bhrase is one headed by a verb. It may be composed of only a single vert 'p, but typically it consists of combinations of main and auxiliary verbs, plus optional specifiers, complements (not including subject complements), and adjuncts. For example: Yankee batters hit the ball well enough to win their first World Series since 2000 Mary saw the man through the window. David gave Mary a book. The first example contains the long verb phrase hit the ball well enough to win their first World Series since 2000; the second is a verb phrase composed of the main verb saw, the Complement phrase the man (a noun phrase), and the adjunct phrase through the window (an adverbial phrase and prepositional phrase). The third example presents three elements, the main verb gave, the noun Mary, and the noun phrase a book, all of which comprise the verb Phrase. Note, the verb phrase described here corresponds to the predicate of traditional grammar. Current views vary on whether all languages have a verb phrase; some schools of ‘generative grammar (such as principles and parameters) hold that all languages have a verb phrase, while others (such as lexical functional qrammar) take the view that at least some languages lack a verb phrase constituent, including those languages with a very free word order (the so-called non- configurational languages, such as Japanese, Hungarian, or Australian aboriginal languages), and ‘some languages with a default VSO order (several Celtic and Oceanic languages) ‘Phrase structure grammars view both finite and nonfinite verb phrases as constituent phrases and, consequently, do not draw any key distinction between them. Dependency:grammars (described below) are much different in this regard, In dependency grammarsjedit} ‘While phrase structure grammars (constituency grammars) fcknowledge both fina and non finite VPs as constituents (complete subtrees), dependency grammars reject former. . dependency grammars acknowledge only non-finite VPs as constituents; finite VPs do not qualify as constituents in dependency grammars. For example: John has finished the work. - Finite VP in bold John has finished the work. - Non-finite VP in boldSince has finished the work contains the finite verb has, itis a finite VP, and since finished the work contains the non-finite verb finished but lacks a VP. Similar examples They do not want to try that. - Finite VP m bold They do not want to try that. — One non-fnite VP in bold They do not want to try that. - Another non-fnite VP in bold finite verb, itis a non-finite These examples illustrate well that many clauses can contain more than one non-finite VP, but they generally contain only one finite VP. Starting with Lucien Tesniére 1969," dependency grammars challenge the validity of the initial binary division of the clause into subject (NP) and predicate (VP), which means they reject the notion that the second half of this binary division ‘Le. the finite VP, is a constituent. They do, however, readily acknowledge the existence of non-finite VPs as constituents. The two competing views of verb phrases are visible in the following trees. s N a. John has finished the work. b. John has finished the work. fi Constituency structure Dependency structure The constituency tree on the left shows the finite VP has finished the work as a constituent, since it corresponds to a complete subtree. The dependency tree on the right, in contrast, does not acknowledge a finite VP constituent, since there is no complete subtree there that corresponds to has finished the work. Note that the analyses agree concerning the non-finite VP finished the work; both see it as a constituent (complete subtree) : Dependency grammars point to the results of many standard constituency tests to back up their stance." For instance, topicalization, pseudoclefting, and answer ellipsis suggest that non-finite VP does, but finite VP does not, exist as a constituent: 1..and has finished the work, John. - Topicalization “What John has done is has finished the work. - Pseudocletting What has John done? ~ *Has finished the work. - Answer ellipsis ‘The * indicates that the sentence is bad. These data must be compared to the results for non-finite VP: ..and finished the work, John (certainly) has. - Topicalization ‘What John has done is finished the work. ~ Pseudoclefting What has John done? — Finished the work. ~ Answer elipsis ‘The strings in bold are the ones in focus. Attempts to in some sense isolate the finite VP fail, but the same attempts with the non-finite VP succeed.” Narrowly defined Verb phrases are sometimes defined more narrowly in scope, in effect counting only those elements considered strictly verbal in verb phrases. That would limit the definition to only main and auxliary verbs, plus infinitive or participle constructions. For example, in the following sentences only the words in bold form the verb phrase: ‘ ‘iJohn has given Mary a book. The picnickers were being eaten alive by mosquitos. She kept screaming like a football maniac. Thou shalt not kill, This more narrow definition is often applied in functionalist frameworks and traditonal European reference grammars. It is incompatible with the phrase structure model, becaces tre strings in bold are not constituents under that analysis. It is, however, compatible with grammars and other grammars that view the verb catena (verb chain) as the fundamental unit of syntactic structure, as opposed to the constituent. ‘Part-of-Speech (POS ying Part-of-Speech (POS) tagging is a fundamental task in Natural Language Processing (NLP) that involves assigning a grammatical category (such as noun, verb, adjective, etc.) to each word ina sentence. The goal is to understand the syntactic structure of a sentence and identify the grammatical roles of individual words. POS tagging provides essential information for various NLP applications, including text analysis, machine translation, and information retrieval. Key Concepts: 1. POS Tags: + POS tags are short codes representing specific parts of speech. Common POS tags include: + Noun (NN) + Verb (VB) + Adjective (JJ) + Adverb (RB) + Pronoun (PRP) + Preposition (IN) + Conjunction (CC) . Determiner (DT) + Interjection (UH) 2.Tag Sets: + Different tag sets may be used depending on the POS tagging system or language. For example, the Penn Treebank POS tag set is widely used in English NLP tasks. -Ambiguity: 2 Words may have multiple possible POS tags based on context. - For example, “lead” can be a noun (the metal) or a verb (to guide). Importance and Applications: 1. Syntactic Analysis: i + _ POS tagging is crucial for understanding the grammatical | ‘ structure of a sentence, enabling syntactic analysis. It helps identify the subject, verb, object, and other syntactic elements. 2,Semantic Analysis:Part-of-Speech (POS) tagging Part-of-Speech (POS) tagging is a fundamental task in Natural Language Processing (NLP) that involves assigning a grammatical category (such as noun, verb, adjective, etc.) to each word in a sentence. The goal is to understand the syntactic structure of a sentence and identify the grammatical roles of individual words. POS tagging provides essential information for various NLP applications, including text analysis, machine translation, and information retrieval. Key Concepts: 1. POS Tags: ¢ + _ POS tags are short codes representing specific parts of speech. Common POS tags include: + Noun (NN) + Verb (VB) + Adjective (JJ) + Adverb (RB) + Pronoun (PRP) + Preposition (IN) * Conjunction (CC) + Determiner (DT) . Interjection (UH) 2.Tag Sets: ss + Different tag sets may be used depending on the POS tagging system or language. For example, the Penn Treebank POS tag set is widely used in English NLP tasks. 3-Ambiguity: + Words may have multiple possible POS tags based on context. For example, “lead” can be a noun (the metal) or a verb (to guide). Importance and Applications: 1. Syntactic Analysis: + POS tagging is crucial for understanding the grammatical i structure of a sentence, enabling syntactic analysis. It helps identify the subject, verb, object, and other syntactic elements. ui 2.Semantic Analysis:+ Words often have multiple possible POS tags based on context, leading to challenges in disambiguation. Sontext Dependency: + POS tags can depend on the surrounding words, making accurate ive to context. ulary Words: + Handling words not seen during training is a challenge, as their POS tags need to be pri .d based on context. Example: Consider the sentence: “The quick brown fox jumps over the lazy dog.” POS tagging might yield: + “The” (DT): Determiner + “quick” (JJ): Adjective “brown” (JJ): Adjective “fox” (NN): Noun “jumps” (VBZ): Verb “over” (IN): Preposition “the” (DT): Determiner “lazy” (JJ): Adjective + “dog” (NN): Noun In this example, each word is assigned a POS tag indicating its grammatical category in the sentence. Applications where POS tagging plays a crucial role: 1. Syntactic Parsing: . Application: Understanding the grammatical structure of sentences. + Role of POS Tagging: POS tags provide information about the syntactic role of each word, aiding in syntactic parsing and tree construction. s 2. Named Entity Recognition (NER): 5 - Application: Identifying and classifying entities (e.g., persons, organizations, locations) in text. + Rolé of POS Tagging: POS tags help in identifying proper nouns, which are often indicative of named entities. 3. Information Retrieval: - Application: Improving search and retrieval of relevant documents or information. + Role of POS Tagging: Using POS tags, one can prioritize or filter search results based on the grammatical category of words. For instance, focusing on nouns for certain queries.3. Information Retrieval: + _ POS tagging is used in information retrieval systems to improve the precision and relevance of search results. For instance, searching for “NN” (noun) in a document can prioritize nouns over other words. 4. Named Entity Recognition (NER): + _ POS tags play a role in named entity recognition by providing information about the grammatical category of words. For example, recognizing that “New York” is a proper noun 5.-Machine Translation: + POS tagging is essential in machine translation to ensure accurate translation based on the grammatical structure of sentences. Methods of POS Tagging: 1. Rule-Based Taggin; + Based on handcrafted rules that consider word morphology, context, and syntactic information. It can be effective but may struggle with ambiguity. 2. Statistical Tagging: + Uses statistical models trained on large annotated corpora to predict POS tags. Hidden Markov Models (HMMs) and Conditional Random Fields (CRFs) are common statistical approaches. 3.Machine Learning-Based Tagging: + Utilizes machine learning algorithms such as decision trees, support vector machines, or neural networks to learn patterns from data. Particular emphasis is given to contextual information. 4.Deep Learning-Based Tagging: + Deep learning models, such as recurrent neural networks (RNNS) and long short-term memory networks (LSTMs), are employed for POS tagging, capturing complex contextual dependencies. Challenge: 1. Ambiguity: + — Words often have multiple possible POS tags based on context, leading to challenges in disambiguation. 2.Context Dependency:fastTextFastText is an open-source, free librar fastTextFas is an open-soure! ary from Facebook Al Research(FAIR} learning word embeddings and word classifications. This model allows eating ae alpen ead learning or supervised learning algorithm for obtaining vector representations for words. It also evaluates these models, FastText st CBOW and Skip-gram models. cee Uses of FastText: 1 It is used for finding semantic similarities 2 It can also be used for text classification(ex: spam filtering). 3 It can train large datasets in minutes NODE O Leaf wa : Travel e Leaf w3 : Indian Leaf w2 : Food pas, 10 Stanford toolkit (Glove) Global Vectors for Word Representation, or GloVe for short, is Sn unsupervised learning algorithm that generates vector representations, or embeddings, Gf words. Researchers Richard Socher, Christopher D. Manning, and Jeffrey Pennington first presented it in 2014. By using the statistical co-occurrence data of words in a given corpus, GloVe is intended to capture the semantic relationships between words. ‘The fundamental concept underlying GloVe is the representation of words as vectors i 8 dontinuous vector space, where the angle and direction of the vectors correspond to the comantic connections between the appropriate words. To do this, GloVe builds. a ts sceurrence matrix using word pairs and then optimizes the word vectors to miimize the Gifference between the pointwise mutual information of the corresponding words and the dot product of vectors.