989 lines
15 KiB
HTML
989 lines
15 KiB
HTML
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
|
<meta name="description" content="">
|
|
<meta name="author" content="Eelco Visser">
|
|
<meta name="generator" content="Jekyll v3.8.5">
|
|
<title>Lecture 8a: From Context-Free Grammars to Parsers</title>
|
|
<!-- <base href="/2021"> -->
|
|
|
|
<!--link rel="canonical" href="https://getbootstrap.com/docs/4.3/examples/starter-template/"-->
|
|
|
|
<link rel="icon" href="../img/logo/pl_ico2_2B3_icon.ico" type="image/x-icon">
|
|
|
|
<!-- Bootstrap core CSS -->
|
|
<!--link href="https://getbootstrap.com/docs/4.3/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous"-->
|
|
|
|
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
|
|
|
|
|
|
|
|
<style>
|
|
.bd-placeholder-img {
|
|
font-size: 1.125rem;
|
|
text-anchor: middle;
|
|
-webkit-user-select: none;
|
|
-moz-user-select: none;
|
|
-ms-user-select: none;
|
|
user-select: none;
|
|
}
|
|
|
|
@media (min-width: 768px) {
|
|
.bd-placeholder-img-lg {
|
|
font-size: 3.5rem;
|
|
}
|
|
}
|
|
</style>
|
|
<!-- Custom styles for this template -->
|
|
<link href="../css/main.css" rel="stylesheet">
|
|
<link href="../css/borders-responsive.css" rel="stylesheet">
|
|
|
|
<link rel="stylesheet" href="../css/pl.css">
|
|
</head>
|
|
<body>
|
|
|
|
<nav class="navbar navbar-expand-md navbar-dark bg-dark fixed-top">
|
|
|
|
<a class="navbar-brand" href="../index.html">
|
|
TU Delft | CS4200
|
|
</a>
|
|
|
|
<button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarsExampleDefault" aria-controls="navbarsExampleDefault" aria-expanded="false" aria-label="Toggle navigation">
|
|
<span class="navbar-toggler-icon"></span>
|
|
</button>
|
|
|
|
<div class="collapse navbar-collapse" id="navbarsExampleDefault">
|
|
<ul class="navbar-nav mr-auto">
|
|
|
|
|
|
|
|
|
|
|
|
<li class="nav-item active">
|
|
<a class="nav-link" href="../lectures/index.html" tabindex="-1" aria-disabled="true">Lectures</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="nav-item active">
|
|
<a class="nav-link" href="../homework/index.html" tabindex="-1" aria-disabled="true">Homework</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="nav-item active">
|
|
<a class="nav-link" href="../project/index.html" tabindex="-1" aria-disabled="true">Project</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="nav-item active">
|
|
<a class="nav-link" href="../news/index.html" tabindex="-1" aria-disabled="true">News</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="nav-item active">
|
|
<a class="nav-link" href="../blog/index.html" tabindex="-1" aria-disabled="true">Blog</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
</div>
|
|
</nav>
|
|
|
|
|
|
<div class="container">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="row">
|
|
<div class="col-sm-12 col-md-12 col-lg-12 col-xl-12">
|
|
|
|
<a href="../lectures/2021/07-parsing/CS4200-2021-7-parsing.pdf">
|
|
<img class="border border-dark" width="100%" src="../lectures/2021/07-parsing/CS4200-2020-7-parsing-slr-table.png"/>
|
|
</a>
|
|
|
|
|
|
<div class="mt-3 mb-3 pt-3 pb-3 text-dark border-top border-bottom border-grey">
|
|
<div class="text-dark font-weight-bold">
|
|
<h1>
|
|
|
|
Lecture 8a: From Context-Free Grammars to Parsers
|
|
|
|
</h1>
|
|
</div>
|
|
<div>
|
|
|
|
Eelco Visser
|
|
|
|
|
|
</div>
|
|
<div>
|
|
|
|
|
|
Lecture
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <a class="text-primary" href="../lectures/2021/07-parsing/CS4200-2021-7-parsing.pdf">PDF</a>
|
|
|
|
|
|
</div>
|
|
<div>
|
|
October 14, 2021
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="row">
|
|
<div class="col-sm-12 col-md-12 col-lg-9 border-lg-right border-grey">
|
|
<p>In this lecture we study how we can derive parsers from context-free grammars.
|
|
We start with looking again at the semantics of context-free grammars as derivations.
|
|
Then we observe that parsing is the inverse of derivation, i.e. reducing a sentence to a (start) symbol or parse tree.
|
|
A shift-reduce parsers computes a right-most derivation for a sentence.
|
|
The question is then how to make that process deterministic and efficient.
|
|
First we see how item sets encode the state of a shift-reduce parser and make it clear what actions can be undertaking without consulting the grammar.
|
|
Next, we compute the FOLLOW set of a grammar in order to determine which action to take next in case there are multiple possible actions in a state.
|
|
This approach yields a linear deterministic parser for grammars in the the SLR(1) format, i.e. those grammars for which this method produces a parse table without shift/reduce conflicts.</p>
|
|
|
|
<!-- <img src="/2021/lectures/2021/07-parsing/CS4200-2020-7-parsing-slr-table.png" style="border: solid 1px black;"/> -->
|
|
|
|
<h3 id="slides">Slides</h3>
|
|
|
|
<ul>
|
|
<li><a href="../lectures/2021/07-parsing/CS4200-2021-7-parsing.pdf">Slides</a></li>
|
|
<li><a href="../lectures/2021/07-parsing/CS4200-2021-7-parsing-builds.pdf">Slides with builds</a></li>
|
|
</ul>
|
|
|
|
<h3 id="references">References</h3>
|
|
|
|
|
|
|
|
|
|
<ul class="list-group list-group-flush pb-3">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="list-group-item pl-lg-0 border-0 pb-0">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="bd-callout border-primary pl-3">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div>
|
|
<a name="KatsVW10"></a>
|
|
|
|
<a class="font-weight-bold text-dark" target="_blank" href="http://doi.acm.org/10.1145/1869459.1869535">Pure and declarative syntax definition: paradise lost and regained</a>
|
|
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
<a class="text-secondary" href="http://www.lclnet.nl/">Lennart C. L. Kats</a>, <a class="text-secondary" href="http://eelcovisser.org">Eelco Visser</a>, <a class="text-secondary" href="https://www.linkedin.com/in/guidowachsmuth/">Guido Wachsmuth</a>.
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
|
|
|
|
OOPSLA 2010
|
|
|
|
|
|
|
|
|
|
|
|
[<a class="text-secondary" href="http://doi.acm.org/10.1145/1869459.1869535" target="_blank">doi</a>, <a class="text-secondary" target="_blank" href="https://researchr.org/publication/KatsVW10/bibtex">bib</a>, <a href="https://researchr.org/publication/KatsVW10" class="text-secondary" target="_blank">researchr</a>, <a class="text-secondary" data-toggle="collapse" href="8.html#AbstractKatsVW10" role="button" aria-expanded="false" aria-controls="AbstractKatsVW10"">abstract</a>]
|
|
|
|
<!-- KatsVW10 -->
|
|
|
|
</div>
|
|
|
|
<div class="collapse mt-1 mb-2" id="AbstractKatsVW10">
|
|
<div class="">
|
|
Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained.
|
|
</div>
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</div>
|
|
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="list-group-item pl-lg-0 border-0 pb-0">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="bd-callout border-info pl-3">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div>
|
|
<a name="AhoLSU2006"></a>
|
|
|
|
Compilers: Principles, Techniques, and Tools (2nd Edition)
|
|
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
<a class="text-secondary" href="https://researchr.org/alias/a.-v.-aho">A. V. Aho</a>, <a class="text-secondary" href="https://researchr.org/alias/m.-s.-lam"> M. S. Lam</a>, <a class="text-secondary" href="https://researchr.org/alias/r.-sethi">R. Sethi</a>, <a class="text-secondary" href="https://researchr.org/alias/j.-d.-ullman"> J. D. Ullman</a>.
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[<a class="text-secondary" target="_blank" href="https://researchr.org/publication/AhoLSU2006/bibtex">bib</a>, <a href="https://researchr.org/publication/AhoLSU2006" class="text-secondary" target="_blank">researchr</a>]
|
|
|
|
<!-- AhoLSU2006 -->
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</div>
|
|
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="list-group-item pl-lg-0 border-0 pb-0">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="bd-callout border-primary pl-3">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div>
|
|
<a name="Visser97-IWPT"></a>
|
|
|
|
A Case Study in Optimizing Parsing Schemata by Disambiguation Filters
|
|
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
<a class="text-secondary" href="http://eelcovisser.org">Eelco Visser</a>.
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
|
|
|
|
International Workshop on Parsing Technology (IWPT 1997) 1997
|
|
|
|
|
|
|
|
|
|
|
|
[<a class="text-secondary" target="_blank" href="https://researchr.org/publication/Visser97-IWPT/bibtex">bib</a>, <a href="https://researchr.org/publication/Visser97-IWPT" class="text-secondary" target="_blank">researchr</a>, <a class="text-secondary" data-toggle="collapse" href="8.html#AbstractVisser97-IWPT" role="button" aria-expanded="false" aria-controls="AbstractVisser97-IWPT"">abstract</a>]
|
|
|
|
<!-- Visser97-IWPT -->
|
|
|
|
</div>
|
|
|
|
<div class="collapse mt-1 mb-2" id="AbstractVisser97-IWPT">
|
|
<div class="">
|
|
Disambiguation methods for context-free grammars enable concise specification of programming languages by ambiguous grammars. A disambiguation filter is a function that selects a subset from a set of parse trees---the possible parse trees for an ambiguous sentence. The framework of filters provides a declarative description of disambiguation methods independent of parsing. Although filters can be implemented straightforwardly as functions that prune the parse forest produced by some generalized, this can be too inefficient for practical applications. In this paper the optimization of parsing schemata, a framework for high-level description of parsing algorithms, by disambiguation filters is considered in order to find efficient parsing algorithms for declaratively specified disambiguation methods. As a case study the optimization of the parsing schema of Earley's parsing algorithm by two filters is investigated. The main result is a technique for generation of efficient LR-like parsers for ambiguous grammars modulo priorities.
|
|
</div>
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</div>
|
|
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="list-group-item pl-lg-0 border-0 pb-0">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="bd-callout border-success pl-3">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div>
|
|
<a name="Sikkel93-thesis"></a>
|
|
|
|
<a class="font-weight-bold text-dark" target="_blank" href="https://www.base-search.net/Record/019acdb54fd1d19c305a35f7d2572a792c9ad3314ad3ae087a66267f15cadd33">Parsing Schemata</a>
|
|
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
<a class="text-secondary" href="https://researchr.org/alias/klaas-sikkel">Klaas Sikkel</a>.
|
|
</div>
|
|
|
|
<div class="font-weight-light text-secondary" style="font-size:90%;">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PhD thesis, University of Twente, Enschede, Netherlands, 1993
|
|
|
|
[<a href="https://tudelft-cs4200.github.io/2021https://research.utwente.nl/files/6041692/Sikkel93thesis.pdf" class="text-secondary" target="_blank">pdf</a>, <a class="text-secondary" href="https://www.base-search.net/Record/019acdb54fd1d19c305a35f7d2572a792c9ad3314ad3ae087a66267f15cadd33" target="_blank">doi</a>, <a class="text-secondary" target="_blank" href="https://researchr.org/publication/Sikkel93-thesis/bibtex">bib</a>, <a href="https://researchr.org/publication/Sikkel93-thesis" class="text-secondary" target="_blank">researchr</a>]
|
|
|
|
<!-- -->
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</div>
|
|
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
</div>
|
|
<div class="col-sm-12 col-md-12 col-lg-3 col-xl-3 " >
|
|
<div class="sticky-top top70 d-none d-lg-block d-xl-block">
|
|
<div class="pb4 mb3 border-bottom border-grey ">
|
|
<nav>
|
|
<ul class="pagination justify-content-left">
|
|
|
|
<li class="page-item">
|
|
<a class="page-link" href="7.html">
|
|
«
|
|
</a>
|
|
</li>
|
|
|
|
<li class="page-item">
|
|
<a class="page-link" href="../lectures/index.html">
|
|
^
|
|
</a>
|
|
</li>
|
|
|
|
<li class="page-item">
|
|
<a class="page-link" href="8b.html">»</a>
|
|
</li>
|
|
|
|
</ul>
|
|
</nav>
|
|
</div>
|
|
<ul id="my_toc" class="toc list-group list-group-flush d-none d-lg-block d-xl-block p-0 ml-0 mt-3">
|
|
<li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="8.html#slides">Slides</a></li>
|
|
<li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="8.html#references">References</a></li>
|
|
</ul>
|
|
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="row">
|
|
<div class="col-12">
|
|
<div class="border-top border-bottom border-grey mt-3 pt-3">
|
|
<nav>
|
|
<ul class="pagination justify-content-center">
|
|
|
|
<li class="page-item">
|
|
<a class="page-link" href="7.html">
|
|
Previous
|
|
</a>
|
|
</li>
|
|
|
|
|
|
<li class="page-item">
|
|
<a class="page-link" href="8b.html">Next</a>
|
|
</li>
|
|
|
|
<li class="page-item">
|
|
<a class="page-link" href="../lectures/index.html">
|
|
Index
|
|
</a>
|
|
</li>
|
|
</ul>
|
|
</nav>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
</div>
|
|
|
|
|
|
|
|
<!-- Optional JavaScript -->
|
|
<!-- jQuery first, then Popper.js, then Bootstrap JS -->
|
|
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
|
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js" integrity="sha384-UO2eT0CpHqdSJQ6hJty5KVphtPhzWj9WO1clHTMGa3JDZwrnQq4sF86dIHNDz0W1" crossorigin="anonymous"></script>
|
|
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js" integrity="sha384-JjSmVgyd0p3pXB1rRibZUAYoIIy6OrQ6VrjIEaFf/nJGzIxFDsf4x0xIM+B07jRM" crossorigin="anonymous"></script>
|
|
|
|
</body>
|
|
</html>
|