commit dd6e4afb133734b3815f37e00e20fd7d3af13fca Author: Jana Dönszelmann Date: Mon Sep 29 16:03:30 2025 +0200 download 2021 site diff --git a/README.md b/README.md new file mode 100644 index 0000000..4943d75 --- /dev/null +++ b/README.md @@ -0,0 +1,11 @@ + +In 2022 I followed this course, compiler construction, by professor Eelco Visser. +I may not have always agreed with him, but that is normal in academics. +I still remember his teaching well, +and I periodically come back to these slides since they're super instructive even when not using Spoofax, +his language workbench research project in which he poored a large part of his life. +[In 2022, only days after my final exam of this course, he unfortunately passed away far too early.](https://www.tudelft.nl/2022/ewi/in-memoriam-eelco-visser-1966-2022) +These are the slides and course website for 2021, since the version of 2022 is no longer online. +As such I wanted to preserve this information, since it looks like it might be lost to time otherwise. +Just to say this again, I only do this to preserve invaluable knowledgethat may otherwise be lost. +I do not mean to infringe on copyright otherwise. diff --git a/blog/2020/08/31/a-new-compiler-course.html b/blog/2020/08/31/a-new-compiler-course.html new file mode 100644 index 0000000..64ebca0 --- /dev/null +++ b/blog/2020/08/31/a-new-compiler-course.html @@ -0,0 +1,448 @@ + + + + + + + + + A New Compiler Construction Course + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

+ A New Compiler Construction Course +

+
+ August 31, 2020 +
+

We’re changing things this year. +Of course, due to Corona the course takes place mostly online. +But we’re also changing things to the contents of the course.

+ +
    +
  • +

    The course used to be split in a theory part and a practical part. This year, we have split the course in a front-end part (CS4200-A) in Q1 and a back-end part (CS4200-B) in Q2. Each course includes theory and practice of that part of the compiler. So the complete course is still 10 ECTS.

    +
    • +
  • +

    We’re introducing a new website.

    +
    • +
  • +

    Most documentation is on the MetaBorg website.

    +
    • +
  • +

    The project changes from MiniJava, a Java subset, to ChocoPy, a Python subset.

    +
    • +
  • +

    There will be less emphasis in the project on testing.

    +
    • +
  • +

    We will not provide a template project (for the front-end course). You set up the language project in Spoofax from scratch.

    +
    • +
  • +

    We will have programming exercises in WebLab.

    +
    • +
+ +
+ +
+
+ +
+ +
+ + +
+
+ + + + + + + + + + + diff --git a/blog/2021/08/27/a-new-language-workbench.html b/blog/2021/08/27/a-new-language-workbench.html new file mode 100644 index 0000000..d66737e --- /dev/null +++ b/blog/2021/08/27/a-new-language-workbench.html @@ -0,0 +1,440 @@ + + + + + + + + + A New Language Workbench + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

+ A New Language Workbench +

+
+ August 27, 2021 +
+

This year will adopt last year’s structure of the course, and adapt it. +In particular, we will keep the ChocoPy project. +Major changes to the course are

+ +
    +
  • +

    We introduce Spoofax 3 (aka Spoofax-PIE), a redesign of the guts of the workbench recombining the familiar meta languages.

    +
    • +
  • +

    We introduce a new website for the Spoofax meta languages at https://www.spoofax.dev/

    +
    • +
  • +

    For the back-end of the compiler (that you develop in Q2) we adopt the nanopass compiler architecture approach.

    +
    • +
  • +

    We introduce Stratego 2, a gradually typed version of the Stratego transformation language.

    +
    • +
+ +
+ +
+
+ +
+ +
+ + +
+
+ + + + + + + + + + + diff --git a/blog/index.html b/blog/index.html new file mode 100644 index 0000000..87b2fd0 --- /dev/null +++ b/blog/index.html @@ -0,0 +1,230 @@ + + + + + + + + + The CS4200 Blog + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+

The CS4200 Blog

+
+ + + + + + + + +
+
+
+

2021

+
+
+
+
+ +
+
+
+

2020

+
+
+
+
+ + + + + +
+ +
+ + + + + + + + + + + diff --git a/contact/index.html b/contact/index.html new file mode 100644 index 0000000..c5b74db --- /dev/null +++ b/contact/index.html @@ -0,0 +1,172 @@ + + + + + + + + + Contact + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+
+
+
Lecturer
+
+ +
+
+ +
+
+
+
Teaching Assistants
+
+ +
+
+ + +
+ +
+ + + + + + + + + + + diff --git a/css/borders-responsive.css b/css/borders-responsive.css new file mode 100644 index 0000000..9633e0b --- /dev/null +++ b/css/borders-responsive.css @@ -0,0 +1,160 @@ +.border-top { + border-top: 1px solid; } + +.border-right { + border-right: 1px solid; } + +.border-bottom { + border-bottom: 1px solid; } + +.border-left { + border-left: 1px solid; } + +.border-top-0 { + border-top: 0 !important; } + +.border-right-0 { + border-right: 0 !important; } + +.border-bottom-0 { + border-bottom: 0 !important; } + +.border-left-0 { + border-left: 0 !important; } + +.border-x { + border-left: 1px solid; + border-right: 1px solid; } + +.border-y { + border-top: 1px solid; + border-bottom: 1px solid; } + +@media (min-width: 576px) { + .border-sm-top { + border-top: 1px solid; } + + .border-sm-right { + border-right: 1px solid; } + + .border-sm-bottom { + border-bottom: 1px solid; } + + .border-sm-left { + border-left: 1px solid; } + + .border-sm-top-0 { + border-top: 0 !important; } + + .border-sm-right-0 { + border-right: 0 !important; } + + .border-sm-bottom-0 { + border-bottom: 0 !important; } + + .border-sm-left-0 { + border-left: 0 !important; } + + .border-sm-x { + border-left: 1px solid; + border-right: 1px solid; } + + .border-sm-y { + border-top: 1px solid; + border-bottom: 1px solid; } } +@media (min-width: 768px) { + .border-md-top { + border-top: 1px solid; } + + .border-md-right { + border-right: 1px solid; } + + .border-md-bottom { + border-bottom: 1px solid; } + + .border-md-left { + border-left: 1px solid; } + + .border-md-top-0 { + border-top: 0 !important; } + + .border-md-right-0 { + border-right: 0 !important; } + + .border-md-bottom-0 { + border-bottom: 0 !important; } + + .border-md-left-0 { + border-left: 0 !important; } + + .border-md-x { + border-left: 1px solid; + border-right: 1px solid; } + + .border-md-y { + border-top: 1px solid; + border-bottom: 1px solid; } } +@media (min-width: 992px) { + .border-lg-top { + border-top: 1px solid; } + + .border-lg-right { + border-right: 1px solid; } + + .border-lg-bottom { + border-bottom: 1px solid; } + + .border-lg-left { + border-left: 1px solid; } + + .border-lg-top-0 { + border-top: 0 !important; } + + .border-lg-right-0 { + border-right: 0 !important; } + + .border-lg-bottom-0 { + border-bottom: 0 !important; } + + .border-lg-left-0 { + border-left: 0 !important; } + + .border-lg-x { + border-left: 1px solid; + border-right: 1px solid; } + + .border-lg-y { + border-top: 1px solid; + border-bottom: 1px solid; } } +@media (min-width: 1200px) { + .border-xl-top { + border-top: 1px solid; } + + .border-xl-right { + border-right: 1px solid; } + + .border-xl-bottom { + border-bottom: 1px solid; } + + .border-xl-left { + border-left: 1px solid; } + + .border-xl-top-0 { + border-top: 0 !important; } + + .border-xl-right-0 { + border-right: 0 !important; } + + .border-xl-bottom-0 { + border-bottom: 0 !important; } + + .border-xl-left-0 { + border-left: 0 !important; } + + .border-xl-x { + border-left: 1px solid; + border-right: 1px solid; } + + .border-xl-y { + border-top: 1px solid; + border-bottom: 1px solid; } } diff --git a/css/main.css b/css/main.css new file mode 100644 index 0000000..ce1d700 --- /dev/null +++ b/css/main.css @@ -0,0 +1,197 @@ +body { + padding-top: 5rem; +} +.starter-template { + padding: 3rem 1.5rem; + text-align: center; +} + +.border-grey { + border-color: + #888 !important; +} + +.border-light-grey { + border-color: + #bbb !important; +} + +.bd-callout { + border-left: 3px solid; +} + +.notice { + border-left: 4px solid lightgray; + padding: 0.2rem 0 0.2rem 0.4rem; +} + +.notice-warning { + background-color: #fcf8e3; + border-color: #f1c40f; +} + +.notice-danger { + background-color: #f2dede; + border-color: #e74c3c; +} + +.notice-success { + background-color: #dff0d8; + border-color: #2ecc71; +} + +.faded { + opacity: 0.5; +} + +/* blog */ + +.blog h1 { + border-bottom : 1px solid #aaa; + padding-bottom: 10px; + font-size: 1.8rem; +} + +.blog h2 { + border-bottom : 1px solid #aaa; + padding-bottom: 10px; + font-size: 1.5rem; +} + +.blog h3 { + border-bottom : 1px solid #aaa; + padding-bottom: 10px; + font-size: 1.3rem; +} + +.blog .datebox { + font-size: 1rem; +} + +.blog img { + max-width : 100%; + height: auto; +} + + +.post img { + max-width : 100%; + height: auto; +} + +.row img { + max-width : 100%; + height: auto; +} + +/* a { + color: #212529 +} */ + + +/* wiki */ + +.wiki h1 { + border-bottom : 0px solid #aaa; + padding-bottom: 5px; + font-size: 1.8rem; +} + +.wiki h2 { + border-bottom : 1px solid #aaa; + padding-bottom: 5px; + font-size: 1.6rem; +} + +.wiki h3 { + border-bottom : 1px solid #aaa; + padding-bottom: 5px; + font-size: 1.4rem; +} + +.wiki .datebox { + font-size: 1rem; +} + +.wiki img { + max-width : 100%; + height: auto; +} + + +.video-container { + position: relative; + padding-bottom: 56.25%; + padding-top: 35px; + height: 0; + overflow: hidden; +} + + +.video-container iframe { + position: absolute; + top:0; + left: 0; + width: 100%; + height: 100%; +} + +/* .anchor::before { + display: block; + content: " "; + margin-top: -80px; + height: 80px; + width: 0px; + visibility: hidden; + pointer-events: none; + z-index: -100; +} */ + +.anchor a { + position: absolute; + left: 0px; + top: -80px; +} +.anchor { + position: relative; +} + + +.section a { + font-weight: bold; +} + +blockquote { + display: block; + border-left : 4pt solid; + border-color: grey; + padding-left: 1em; +} + +.toc a { + color : black; + font-weight: bold; +} + +.toc-sub a { + color : black; + font-weight: normal; +} + +html { + scroll-padding-top: 70px; /* height of sticky header */ +} + +.top70 { + top : 70px; +} + +div .highlight { + padding-left : 0.8em; + padding-top: 0.1em; + padding-bottom: 0em; + font-family: JetBrains Mone; +} +.highlight { + background-color: #EEE; +} diff --git a/css/pl.css b/css/pl.css new file mode 100644 index 0000000..c02527d --- /dev/null +++ b/css/pl.css @@ -0,0 +1,121 @@ + +/*Navbar*/ +.navbar, .tudelft { + /* background-color: #007bc0; */ + background-color: #1d3056; + /* background-color: #00A6D6; */ +} + +/* +@media(max-width: 992px) { + + .navbar { + min-height: 80px; + } + + .navbar-toggler-icon { + width: 2em; + height: 2em; + } + + body { + font-size: 1.5em; + } + +} + +@media(max-width: 992px) { + .logo { + height: 45px; + } +} +*/ + + +.logo { + height: 35px; +} + + +.navbar-light .navbar-nav .nav-link, .tudelft { + color: #fff; +} + +.navbar-light .navbar-nav .nav-link:focus, .navbar-light .navbar-nav .nav-link:hover { + color: #de1b2c; +} + +/*Content*/ + +/* + +.container-center { + text-align: center; +} + +@media(min-width: 992px) { + .content { + width: 80%; + } +} +*/ + +.content { + padding: 1rem; + display: inline-block; + text-align: left; +} + + + + +/********* Simple or original overlay *******/ +/* Main container */ +.overlay-image { + position: relative; + width: 100%; +} +/* Original image */ +.overlay-image .image { + display: block; + width: 100%; + height: auto; +} +/* Original text overlay */ +.overlay-image .text { + color: #fff; + font-size: 30px; + line-height: 1.5em; + text-shadow: 2px 2px 2px #000; + text-align: center; + position: absolute; + top: 50%; + left: 50%; + transform: translate(-50%, -50%); + width: 100%; +} + +/********* Overlay on hover *******/ +/* New overlay on hover */ +.overlay-image .hover { + position: absolute; + top: 0; + height: 100%; + width: 100%; + opacity: 0; + transition: .5s ease; +} +/* New overlay appearance on hover */ +.overlay-image:hover .hover { + opacity: 1; +} + +/* Make card pictures equal size */ +.people-card img.card-img-top { + height: 350px; + object-fit: cover; + object-position: 50% 0%; +} +@media (max-width: 1199px){.people-card img.card-img-top { height: 400px; } } +@media (max-width: 991px){ .people-card img.card-img-top { height: 450px; } } +@media (max-width: 767px){ .people-card img.card-img-top { height: 600px; } } diff --git a/homework/1-1-a.html b/homework/1-1-a.html new file mode 100644 index 0000000..b6627da --- /dev/null +++ b/homework/1-1-a.html @@ -0,0 +1,273 @@ + + + + + + + + + Week 1.1(a): Compilers + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.1(a): Compilers + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ September 02, 2021 +
+
+
+
+ +
+
+

The assignments for Lecture 1 are essay questions, which you can find on WebLab. +We will not grade these assignments, and they are not part of the Homework grade. +But expect these types of questions on the exam.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/1-1-b.html b/homework/1-1-b.html new file mode 100644 index 0000000..17992cd --- /dev/null +++ b/homework/1-1-b.html @@ -0,0 +1,291 @@ + + + + + + + + + Week 1.1(b): Syntax Definition + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.1(b): Syntax Definition + +

+
+
+ + Eelco Visser + + +
+
+ + + Homework + + + + + + + +
+
+ September 03, 2021 +
+
+
+
+ +
+
+

In this set of assignments, published on WebLab, we exercise syntax definition in SDF3 to define the structure and notation of programs following the material of Lecture 2.

+ + + +

The template for the assignments this week is: given this (fragment of a) language, provide a syntax definition for it. +We give a couple of examples. +In the exam we will test your skills in this department. +So, do exercise your understanding of syntax definition by creating syntax definitions for all the languages you encounter.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/1-1.html b/homework/1-1.html new file mode 100644 index 0000000..0d3ea1c --- /dev/null +++ b/homework/1-1.html @@ -0,0 +1,295 @@ + + + + + + + + + Week 1.2: Disambiguation + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.2: Disambiguation + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ September 09, 2021 +
+
+
+
+ +
+
+

In this set of assignments, published on WebLab, we exercise the disambiguation of grammars following the material of Lecture 3.

+ + + +

The template for the assignments this week is: given this (ambiguous) context-free grammar:

+ +
    +
  • +

    Show that the grammar is indeed ambiguous by giving a sentence and at least two different left-most derivations for that sentence and the corresponding abstract syntax trees.

    +
    • +
  • +

    Disambiguate the grammar using declarative disambiguation rules to indicate the associativity and priority of operators.

    +
    • +
  • +

    Transform the grammar to an unambiguous grammar with equivalent abstract syntax trees for sentences.

    +
    • +
+ +

We also provide an assignment to exercise disambiguation using layout constraints.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/1-3.html b/homework/1-3.html new file mode 100644 index 0000000..0713cd6 --- /dev/null +++ b/homework/1-3.html @@ -0,0 +1,276 @@ + + + + + + + + + Week 1.3: Type Constraints + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.3: Type Constraints + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ September 16, 2021 +
+
+
+
+ +
+
+

In this set of assignments, published on WebLab, we exercise the definition of simple type constraints in Statix following the material of Lecture 4.

+ + + +

The template for the assignments this week is: given this language (the syntax definition of which you can find in the ‘test data’ of the assignment) define the given predicate (e.g. typeOfExp) for all the constructs of the language such that all tests succeed. The assignment text gives a high-level explanation of the expected properties of operators. The tests provide concrete examples.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/1-4.html b/homework/1-4.html new file mode 100644 index 0000000..66c0736 --- /dev/null +++ b/homework/1-4.html @@ -0,0 +1,271 @@ + + + + + + + + + Week 1.4: Name Binding and Name Resolution + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.4: Name Binding and Name Resolution + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ September 23, 2021 +
+
+
+
+ +
+
+

In this set of assignments, published on WebLab, we exercise the definition of name resolution in Statix following the material of Lecture 5.

+ + + +

The template for the assignments this week is: given this language (the syntax definition of which you can find in the ‘test data’ of the assignment) define the given predicate (e.g. typeOfExp) for all the constructs of the language such that all tests succeed. The assignment text gives a high-level explanation of the expected properties of operators. The tests provide concrete examples.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/1-5.html b/homework/1-5.html new file mode 100644 index 0000000..6eb0d28 --- /dev/null +++ b/homework/1-5.html @@ -0,0 +1,262 @@ + + + + + + + + + Week 1.5: Constraint Resolution + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.5: Constraint Resolution + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ September 30, 2021 +
+
+
+
+ +
+
+

In this set of assignments, published on WebLab, we exercise constraint resolution techniques following the material of Lecture 6.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/1-7.html b/homework/1-7.html new file mode 100644 index 0000000..674c9d3 --- /dev/null +++ b/homework/1-7.html @@ -0,0 +1,257 @@ + + + + + + + + + Week 1.6: Parsing + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Week 1.6: Parsing + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ October 08, 2021 +
+
+
+
+ +
+
+

In this set of assignments, published on WebLab, we exercise the material of Lecture 7 and Lecture 8 on parsing algorithms.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/homework/index.html b/homework/index.html new file mode 100644 index 0000000..d1b67d8 --- /dev/null +++ b/homework/index.html @@ -0,0 +1,439 @@ + + + + + + + + + Homework Assignments + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+

Homework Assignments

+
+ See also: + Lectures + | Project +
+
+
+Note that last year's assignments are list marked with (2020). They will be updated to their definitive 2021 versions. +
+ + + + + + + + +
+
+
+

2021

+
+
+
+
+ + + + + +
+ +
+ + + + + + + + + + + diff --git a/homwork/2-1.html b/homwork/2-1.html new file mode 100644 index 0000000..a99f286 --- /dev/null +++ b/homwork/2-1.html @@ -0,0 +1,239 @@ + + + + + + + + + Homework Q2 + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Homework Q2 + +

+
+
+ + +
+
+ + + Homework + + + + + + + +
+
+ November 18, 2021 +
+
+
+
+ +
+
+

See WebLab for further homework assignments to prepare for the exam.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/img/logo/PL-logo72.png b/img/logo/PL-logo72.png new file mode 100644 index 0000000..081b392 Binary files /dev/null and b/img/logo/PL-logo72.png differ diff --git a/img/logo/pl_ico2_2B3_icon.ico b/img/logo/pl_ico2_2B3_icon.ico new file mode 100644 index 0000000..ac5318b Binary files /dev/null and b/img/logo/pl_ico2_2B3_icon.ico differ diff --git a/img/spoofax/Spoofax-Screenshot2.png b/img/spoofax/Spoofax-Screenshot2.png new file mode 100644 index 0000000..5ffb33a Binary files /dev/null and b/img/spoofax/Spoofax-Screenshot2.png differ diff --git a/img/vmb6/2019/2019-03-27-building-28.jpg b/img/vmb6/2019/2019-03-27-building-28.jpg new file mode 100644 index 0000000..3ac43db Binary files /dev/null and b/img/vmb6/2019/2019-03-27-building-28.jpg differ diff --git a/img/weblab/WebLabScreenshot2.png b/img/weblab/WebLabScreenshot2.png new file mode 100644 index 0000000..e820163 Binary files /dev/null and b/img/weblab/WebLabScreenshot2.png differ diff --git a/index.html b/index.html new file mode 100644 index 0000000..021426f --- /dev/null +++ b/index.html @@ -0,0 +1,761 @@ + + + + + + + + + CS4200 | Compiler Construction | 2021-2022 + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+
+
+ +
+
+ +
+
+

TU Delft | CS4200 | 2021-2022

+

Compiler Construction

+
+
+ +
+
    +
  • + Master Computer Science +
    • +
  • + Common Core Software Technology +
    • +
  • + Specialization Programming Languages +
    • +
  • + Credits: 2 x 5 ECTS +
    • +
+
+ +
+
+ Schedule +
+
+ +
+
    +
  • + Lectures Q1: Thursday 11:00 - 12:45 +
    • +
  • + Lab Q1: Tuesday 10:45 - 12:45, 13:45 - 15:45 +
    • +
  • + Lectures Q2: Thursday 15:45 - 17:45 +
    • +
  • + Lab Q2: Friday 13:45 - 17:45 +
    • +
+
+ +
+
+ Organization +
+
+ + + + +
+ +
+ +
+
+ Course Staff +
+
+ +
+
    +
  • + Lecturer: Eelco Visser + (e.visser@tudelft.nl) +
    • +
  • + TA: Ruben van Baarle +
    • +
  • + TA: Thijs Molendijk +
    • +
  • + TA: Aron Zwaan (for Q1) +
    • +
  • + TA: Jeff Smits (for Q2) +
    • +
  • + TA: Gabriël Konat (for Spoofax3) +
    • +
+
+ +
+ + + + +

Computer Science Building 28 at Van Mourik Broekmanweg

+ +
+ +
+ +
+ + + +
+
+
+ Compiler Construction +
+ +

Compilers translate the source code of programs in a high-level programming language into executable (virtual) machine code. Nowadays, compilers are typically integrated into development environments providing features like syntax highlighting, content assistance, live error reporting, and continuous target code generation.

This course studies the architecture of compilers and interactive programming environments and the concepts and techniques underlying the components of that architecture. For each of the components of a compiler we study the formal theory underlying the language aspect that it covers, declarative specification languages to define compiler components, and the techniques for their implementation. The concepts and techniques are illustrated by application to small languages or language fragments.

The compiler construction program consists of two courses of 5 ECTS each.

+ + + + +
+
+ + + + + + +
+
+
+ CS4200-A: Front-End (Q1) +
+ +

In the first course we study the front-end of the compiler that parses and type checks programs. We study meta-languages for the definition of the syntax and static semantics of programming languages. [Study guide]

+ + + + +
+
+ + + + + + +
+
+
+ CS4200-B: Back-End (Q2) +
+ + In the second course we study the back-end of the compiler that performs static analysis, optimization, and code generation. [Study guide] + + + + +
+
+ + + + + + +
+
+
+ Spoofax Language Workbench +
+ + + + + + + + The Spoofax Language Workbench is an environment for the development of programming languages using high-level declarative meta-languages for syntax, static semantics, dynamic semantics, program analysis, and program transformation. We use Spoofax to implement a compiler and IDE for the project language and in homework assignments to experiment with various aspects of compilers in isolation. + + + + +
+
+ + + + + + +
+
+
+ ChocoPy: Project Language +
+ + In the course project students implement a compiler for ChocoPy, a subset of Python3 with static types. The back-end of the compiler generates RISC-V code, which can be run with a simulator running on the JVM. The language comes with a fully specified using formal grammar, typing rules, and operational semantics. ChocoPy was developed at UC Berkeley for their compiler course. + + + + +
+
+ + + + + + +
+
+
+ WebLab: Homework Assignments and Exams +
+ + + + + + + + We use the WebLab learning management system for homework assignments. In WebLab you can submit answers to exercises with grammars, type systems, and transformations, which we can automatically grade. You can even program the solutions directly in WebLab, but it is probably more convenient at this stage to do that in Spoofax and then copy the solutions. We will also use WebLab for the (online) digital exams for the course. Thus, you will be expected to program in the Spoofax meta-languages at the exam. + + + + +
+
+ + + + + + +
+
+ Lectures + | Project + | Homework + | WebLab + | Spoofax + | ChocoPy +
+
+ +
+ +
+ +
+ +
+ +
+
+

Tentative Lecture Schedule CS4200-A/Q1

+

+ Lectures are on Thursdays 10:45 - 12:45. +

+
+ +
+
+ +
+
+
+

Tentative Lecture Schedule CS4200-B/Q2

+

+ Lectures are on Thursdays 10:45 - 12:45. (Note the extra lecture on Friday Sep 4.) +

+
+ +
+
+
+ +
+
+
+
+

Recorded Talks about Spoofax

+
+
+ +
+
+ + A YouTube Playlist with recordings of tutorials and research talks about (research related to) the Spoofax Language Workbench. + +
+
+
+
+ + + + + + + +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/0.html b/lab/0.html new file mode 100644 index 0000000..fe6ae58 --- /dev/null +++ b/lab/0.html @@ -0,0 +1,414 @@ + + + + + + + + + Rules for the Course Project + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + + Rules for the Course Project + + +

+
+
+ + +
+
+ + + + Project Rules + + + + + + + + +
+
+ September 01, 2021 +
+
+
+
+ +
+
+

Submission

+ +

We rely on GitLab for project assignment submissions. We assign a private GitLab repository to each student and encourage students to commit and push their work frequently.

+ +
    +
  • To submit your assignment, you need to file a merge request. There are two kinds of submissions.
    • +
  • To check your progress on an assignment, you can submit a preliminary solution. We will provide limited feedback on preliminary solutions. This feedback typically comes with a tentative grade and points out areas where your solution is incomplete or insufficient, without giving any details on the reasons.
    • +
  • To submit your solution for grading, you need to submit a final solution. As the name suggests, you cannot submit another final solution for the same assignment. +We will provide a detailed grading report on final solutions. To ensure fairness and equal chances for all students, grading reports will not be published before 2 days after the submission deadline. We do our best to hand out grading reports as quick as possible, but manual grading can be tedious and might result in longer waiting times for some assignments.
    • +
+ +

Grades

+ +

We grade the first final solution of each of your assignments. Grades reflect on correctness, clarity, shown programming skills, and readability of your submission. To pass the practical part of the course, you need to meet all of the following criteria:

+ +
    +
  1. You completed each assignment with a grade of 4.0 or better.
    2. +
  2. You completed the assignments of each milestone with an average grade of 5.0 or better.
    3. +
  3. You completed all the assignments with an average grade of 6.0 better.
    4. +
+ +

These rules allow you to compensate for lower grades in single assignments, but ensure a minimal quality in all your milestones as well as in your overall practical work.

+ +

Deadlines

+ +

Assignments must be submitted by the published deadline. You get a standard deadline extension of 48 hours with a penalty of 2 grade points computed proportionally to the amount of extra time you use.

+ +

If you think you cannot meet deadlines due to special personal circumstances, you should contact the academic counsellor. Together we will try to find an individual solution for you. Remember to contact the academic counsellor as soon as possible, once you become aware of your situation. Otherwise, you might lose your chance for a deadline extension.

+ +

Academic Misconduct

+ +
    +
  • All actual, detailed work on assignments must be individual work.
    • +
  • You are encouraged to discuss assignments, programming languages used to solve the assignments, their libraries, and general solution techniques in these languages with each other. If you do so, then you must acknowledge the people with whom you discussed at the top of your submission.
    • +
  • You should not look for assignment solutions elsewhere; but if material is taken from elsewhere, then you must acknowledge its source.
    • +
  • You are not permitted to provide or receive any kind of solutions of assignments. This includes partial, incomplete, or erroneous solutions.
    • +
  • You are also not permitted to provide or receive programming help from people other than the teaching assistants, PhD students associated with this course, or the course manager.
    • +
+ +

Any violation of these rules will be reported as a suspected case of fraud to the Board of Examiners and handled according to the EEMCS Faculty’s fraud procedure. If the case is proven, a penalty will be imposed: a minimum of exclusion from the course for the duration of one academic year up to a maximum of a one-year exclusion form all courses at TU Delft. For details on the procedure, see Section 2.1.26 in the faculty’s Study Guide for MSc Programmes.

+ +

Retry

+ +

To ensure fairness and equal chances for all students, you are not allowed to take the same practical assignments in another edition of this course. In case you participated in previous editions of the course, please contact the course manager to discuss an individual assignment.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/0a.html b/lab/0a.html new file mode 100644 index 0000000..d141662 --- /dev/null +++ b/lab/0a.html @@ -0,0 +1,1020 @@ + + + + + + + + + Lab 0(a): The ChocoPy Language + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + + Lab 0(a): The ChocoPy Language + + +

+
+
+ + +
+
+ + + + Project Language Documentation + + + + + + + + | PDF + + +
+
+ September 03, 2021 +
+
+
+
+ +
+
+

Project Language

+ +

In the course project you will build a compiler and IDE for the ChocoPy language. Quoting from the ChocoPy website:

+ +
+

ChocoPy is a programming language designed for classroom use in undergraduate compilers courses. ChocoPy is a restricted subset of Python 3, which can easily be compiled to a target such as RISC-V. The language is fully specified using formal grammar, typing rules, and operational semantics. ChocoPy is used to teach CS 164 at UC Berkeley. ChocoPy has been designed by Rohan Padhye and Koushik Sen, with substantial contributions from Paul Hilfinger.

+ +

At a glance, ChocoPy is:

+ +

Familiar: ChocoPy programs can be executed directly in a Python (3.6+) interpreter. ChocoPy programs can also be edited using standard Python syntax highlighting.

+ +

Safe: ChocoPy uses Python 3.6 type annotations to enforce static type checking. The type system supports nominal subtyping.

+ +

Concise: A full compiler for ChocoPy be implemented in about 12 weeks by undergraduate students of computer science. This can be a hugely rewarding exercise for students.

+ +

Expressive: One can write non-trivial ChocoPy programs using lists, classes, and nested functions. Such language features also lead to interesting implications for compiler design.

+ +

Bonus: Due to static type safety and ahead-of-time compilation, most student implementations outperform the reference Python implementation on non-trivial benchmarks.

+
+ +

The ChocoPy Reference Manual

+ +

The ChocoPy reference manual is the authority on the definition of the syntax and semantics of ChocoPy. +Start reading the document to understand the language design.

+ +

ChocoPy Example Programs

+ +

You can find examples of ChocoPy programs at https://github.com/cs164berkeley.

+ +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Rohan Padhye, Koushik Sen, Paul Hilfnger, Rohan Padhye, Koushik Sen, Paul Hilfnger. +
    + +
    + + + + + + + [pdf, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Padhye, Rohan, Sen, Koushik, Hilfinger, Paul N.. +
    + +
    + + + Proceedings of the 2019 ACM SIGPLAN Symposium on SPLASH-E 2019 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + ChocoPy is a programming language designed for teaching an undergraduate course on programming languages and compilers. ChocoPy is a restricted subset of Python 3.6, using static type annotations to enforce compile-time type safety. ChocoPy is fully specified using formal grammar, typing rules, and operational semantics. Valid ChocoPy programs can be executed in a standard Python interpreter, producing results consistent with ChocoPy semantics. A major component of CS164 at UC Berkeley is the project: students develop a full compiler for ChocoPy, targeting RISC-V, in about twelve weeks. In other exercises, students extend the syntax, type system, and formal semantics to support additional features of Python. In this paper, we outline (1) the motivations for creating the ChocoPy project, (2) salient features of the language, (3) the resources provided to students to develop their compiler, (4) some insights gained from teaching two semesters of ChocoPy-based courses by different instructors. Our assignment resources are available for re-use by other instructors and institutions. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + + (editor). +
    + +
    + + + + + https://chocopy.org/ 2019 + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/0b.html b/lab/0b.html new file mode 100644 index 0000000..1d0f575 --- /dev/null +++ b/lab/0b.html @@ -0,0 +1,379 @@ + + + + + + + + + Lab 0(b): Registering Your Project + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + + Lab 0(b): Registering Your Project + + +

+
+
+ + +
+
+ + + + WebLab + + + + + + + + +
+
+ September 03, 2021 +
+
+
+
+ +
+
+

We use the Git version control system with https://gitlab.ewi.tudelft.nl to manage submissions and grade assignments. You can access EWI GitLab with you NetID.

+ +

To setup your individual repository, we need to know your NetID. Please provide it in your answer to this question on WebLab. If your GitLab username is different than your NetID, provide it in your answer too.

+ +

If you are in doubt about your answer, check the model answer by clicking the A button in the top right corner.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/10.html b/lab/10.html new file mode 100644 index 0000000..8dbb048 --- /dev/null +++ b/lab/10.html @@ -0,0 +1,503 @@ + + + + + + + + + Lab 10: Instruction Selection + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 10: Instruction Selection + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ November 26, 2021 +
+
+
+
+ +
+
+

See the slides of Lecture 12 on code generation for an approach to get started.

+ +

Objectives

+ +
    +
  1. +

    Develop tests for your compiler that explore the edge cases.

    +
    2. +
  2. +

    Define a transformation strategy that translates ChocoPy expressions to RISC-V code, including variable definitions, integer and boolean constants and operators.

    +
    3. +
  3. +

    Develop a simplification transformation that transforms expressions such that more concise code can be generated.

    +
    4. +
+ +

Basic Compiler Pipeline

+ +

Follow the slides of Lecture 12 to set up a basic compiler pipeline:

+ +
  compile-to-rv32im-ast :: Program -> RProgram
+
+  compile-to-rv32im-ast =
+    compile-cpy-to-cir
+    ; compile-cir-to-rv32im
+    ; compile-rv32im
+
+
+ +

Debugging

+ +

Use debug(!msg) or dbg(|msg) to print the current term to the console.

+ +

You can also debug your generated code, by creating a .cpy file, transforming it to an .rv32im file using the Spoofax > Generation menu, and copying the generated output to the online Venus editor, where you can use the Simulator tab to run your code or step through each line, and inspect the memory and registers.

+ +

From ChocoPy to C-IR

+ +

Develop a first compilation phase to transform ChocoPy programs to an intermediate language that makes control flow explicit.

+ +
 compile-cpy-to-cir :: Program -> CProgram
+
+ compile-cpy-to-cir =
+   explicate-types
+   ; desugar
+   ; uniquify
+   ; remove-complex-operands
+   ; explicate-control
+
+ +

Explicate Types

+ +

A useful transformation is to type specialize the constructors of the AST, such that the analysis results are no longer needed. +That is useful when applying transformations, since preserving those annotations cannot be done for all transformations. +The following explicate-types transformation should be invoked on the AST before invoking the compiler transformation.

+ +
signature
+  constructors
+    AddInt : Exp * Exp -> Exp
+
+rules
+
+  explicate-types =
+    innermost(type-specialize)
+
+  type-specialize :
+    add@Add(e1, e2) -> AddInt(e1, e2)
+    where <get-type> add => Int()
+
+ +

Note that the rules for translating integer additions should be adapted to reflect the change in constructors.

+ +

Desugar

+ +

To improve the result of code generation, it can be useful to transform the source language expression. +For example, left-associative additions may produce a better result.

+ +
  desugar-exp :
+    AddInt(e1, AddInt(e2, e3)) -> AddInt(AddInt(e1, e2), e3)
+
+ +

Define a transformation to desugar ChocoPy programs, making the task of downstream compilation easier.

+ +

This simplification can include constant folding, eventually. +However, do not yet include constant folding rules, since you want to test the translation of operators. +That is, your code generator should be able to translate arbitrary combinations of operators, so be prepared for the general case.

+ +

Think about (and try out in the online compiler!) the differences between adding integers and concatenating strings in RISC-V. In ChocoPy, they initially both use the Add(...) constructor, so make sure to disambiguate them into separate constructors you will later use for code generation. The same holds for other operations, but you have to think about those yourself.

+ +

Uniquify

+ +

Define a transformation that eliminates shadowing. (See slides)

+ +

Remove Complex Operands

+ +

To simplify instruction selection, simplify expressions to only use atomic expressions (constants or variables) creating new variables to store intermediate results.

+ +

Explicate Control using C-IR

+ +

Use an intermediate language to explicate control. +That is, the explicate-control strategy translates from ChocoPy ASTs

+ +
signature
+  sorts CID CINT CProgram CBlock CLabel CTail CStmt
+        CType CExp CAtom CVar
+  constructors
+                     : string -> CID
+                     : string -> CINT
+    CProgram         : List(CBlock) -> CProgram
+    CBlock           : CLabel * CTail -> CBlock
+    CLabel           : CID -> CLabel
+    CReturn          : CExp -> CTail
+    CReturnNone      : CTail
+    CSeq             : CStmt * CTail -> CTail
+    CVarDec          : CVar * CType * CExp -> CStmt
+    CAssign          : CVar * CExp -> CStmt
+    CIntT            : CType
+                     : CAtom -> CExp
+    CMin             : CAtom -> CExp
+    CAdd             : CAtom * CAtom -> CExp
+    CMul             : CAtom * CAtom -> CExp
+    CDiv             : CAtom * CAtom -> CExp
+    CInt             : CINT -> CAtom
+                     : CVar -> CAtom
+    CVar             : CID -> CVar
+
+ +

Instruction Selection

+ +

Define a transformation from C-IR to RISC-V instructions:

+ +
 compile-cir-to-rv32im :: CProgram -> RProgram
+
+ compile-cir-to-rv32im =
+   select-instructions-cprogram
+
+ +

The essence of this translation consists of instruction selection, i.e. mapping the high-level instructions of the intermediate language to concrete RISC-V instructions. +For example, register allocation for integer constants, variables, and addition may be defined as follows:

+ +
  select-instrs-exp(|RArg) :: CExp -> List(RLine)
+
+  select-instrs-exp(|x) :
+    CInt(i) -> [RLi(x, <cint-to-rint>i)]
+
+  select-instrs-exp(|x) :
+    CVar(y) -> [RMv(x, RVar(<cid-to-string>y))]
+
+  select-instrs-exp(|x) :
+    CAdd(y@CVar(_), z@CVar(_)) -> [RAdd(x, <cvar-to-rvar>y, <cvar-to-rvar>z)]
+
+ +

Note that because expressions have been transformed to applications of operators to atomic expressions, this step does not need to invent names for intermediate results. +We use RVar(x) as symbolic registers in RISC-V code in order to postpone the mapping to concrete registers.

+ +

Define transformation rules for all operators of ChocoPy expressions.

+ +

Special Cases

+ +

RISC-V provides specialized instructions for some operations. +For example, the addi instruction allows directly adding an integer constant (between -2048 and 2047) to a register. +A compiler can make use of such instructions, by detecting special patterns in the source language. +For example, the following rule, detects additions with an integer constant, and translates those to applications of addi, avoiding the use of an extra register.

+ +
select-instrs-exp(|x) :
+  CAdd(z@CVar(_), CInt(i)) -> [RAddi(x, <cvar-to-rvar>z, <cint-to-rint>i)]
+  with debug(!"select-instrs-exp: ")
+
+ +

Can you detect other specialized instructions and corresponding source language patterns that provide a more concise and/or faster target code?

+ +

Booleans

+ +

In ChocoPy we compile the Booleans True and False to integers in RISC-V (1 and 0, respectively). +Implement the Boolean operators and, or and not, and integer comparison operators ==, !=, <, >, <=, >= (you can ignore is for now since it operates on objects).

+ +

You can make use of the online compiler, or the RISC-V instruction set to find the proper instructions in RISC-V.

+ + + +

Patching RISC-V instructions

+ +

Finally, we define a compiler stage that patches up the RISC-V code:

+ +
 compile-rv32im :: RProgram -> RProgram
+
+ compile-rv32im =
+   assign-homes
+   ; patch-instructions
+
+ +

Assign Homes

+ +

Map symbolic registers to concrete registers. +Either by mapping variables to actual registers or by using a stack discipline. +We will look at register allocation next, so keep it simple.

+ +

Patch Instructions

+ +

This is the place for any mopping up that needs to be done.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/11.html b/lab/11.html new file mode 100644 index 0000000..2a6405d --- /dev/null +++ b/lab/11.html @@ -0,0 +1,644 @@ + + + + + + + + + Lab 11: Implementing Register Allocation and Control Flow + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 11: Implementing Register Allocation and Control Flow + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ December 03, 2021 +
+
+
+
+ +
+
+

Register Allocation

+ +

In this lab you implement register allocation taking the implementation from the slides as basis.

+ +

Liveness Analysis

+ +

Implement liveness analysis to determine when (at what instructions) variables are live. +Extend the analysis to deal with control flow and loops. +This requires a fixed point analysis.

+ +

Graph Coloring

+ +

Next implement register allocation bases on the outcome of the liveness analysis. +Use the priority queue data structure to represent the interference graph with the nodes representing variables with the most inference listed first. +Coloring the graph with a limited number of registers. +How do you treat spilling variables to memory?

+ +

Control Flow

+ +

Extend the fragment of the language that is supported by your compiler with control-flow constructs such as if-then-else and booleans.

+ +

Desugaring

+ +

Define desugaring rules to reduce the number of constructs that the downstream compiler has to deal with. +Consider whether or for which language constructs this is worthwhile; the target machine may have instructions that correspond directly to the source language operators.

+ +

Explicating Control: Translation to C-IR

+ +

Translate the recursive expressions for control flow in ChocoPy to jumps in the control flow graph.

+ +

Instruction Selection

+ +

Translate to RISC-V instructions.

+ + + + + + + + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/12.html b/lab/12.html new file mode 100644 index 0000000..11f6c83 --- /dev/null +++ b/lab/12.html @@ -0,0 +1,486 @@ + + + + + + + + + Lab 12: Loops & Functions + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 12: Loops & Functions + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ December 10, 2021 +
+
+
+
+ +
+
+

While Loops

+ +

Extend your compiler to generate code for while loops. +Remember to extend liveness analysis to cater for loops. +That is, there may be multipe blocks that jump to a block. +This requires that the analysis performs a fixpoint computation.

+ +

Functions

+ +

Extend your compiler to generate code for functions. +To cater for nested functions, you should pass a pointer to the call frame to the lecically enclosing function (a so called static link). +Note that the static link may not point to the caller of the function.

+ + + + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/13.html b/lab/13.html new file mode 100644 index 0000000..85a6a16 --- /dev/null +++ b/lab/13.html @@ -0,0 +1,394 @@ + + + + + + + + + Lab 13: Compiling Objects + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 13: Compiling Objects + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ December 17, 2021 +
+
+
+
+ +
+
+

Develop code generation for ChocoPy classes and methods.

+ +

Objectives

+ +
    +
  1. Compiling objects
    2. +
  2. Compiling lists
    3. +
  3. Challenges +
      +
    • Garbage collection
      • +
    • Register allocation
      • +
    • Peephole optimizations
      • +
    +
    4. +
+ +

Compiling Objects

+ +

The main objective for today’s lab is to extend your compiler to support the compilation of classes. +See also the slides of Lecture 15 and the ChocoPy implementation guide.

+ +

Objects in ChocoPy

+ +

ChocoPy supports object-oriented programming through classes with attributes and methods. +The following example program (from the reference manual) illustrates many of the features of classes in ChocoPy:

+ +
class animal(object):
+  makes_noise:bool = False
+
+  def make_noise(self: "animal") -> object:
+    if (self.makes_noise):
+      print(self.sound())
+
+  def sound(self: "animal") -> str:
+    return "???"
+
+class cow(animal):
+  def __init__(self: "cow"):
+    self.makes_noise = True
+
+  def sound(self: "cow") -> str:
+    return "moo"
+
+c:animal = None
+c = cow()
+c.make_noise()
+
+ +

A class defines a type and inherits from a superclass of which it is a subtype.

+ +

A class has attributes that are initialized with a literal value, which is None in the case of object types.

+ +

A class has method defintions, which are regular ChocoPy function definitions, except that methods always have a first argument (self) of the enclosing class type, which represents the objects to which the method is applied. +Methods can have nested functions.

+ +

A method call e0.f(e1, ...) invokes a method on the object. The actual implementation called corresponds to the dynamic type of the object value. That is, when the object is of a subtype t of the static type of e0, and the method f has been overridden in t or a supertype of t, the method lowest in the class hierarchy is called. This is known as dynamic dispatch.

+ +

A function with the same name as the class (e.g. cow() in the example above) is used to create objects. +When a class or one of its superclasses has an __init__ method, that method is called implicitly on construction of the object.

+ +

Object Layout and Prototypes

+ +

An object value is represented by a chunk of memory in the heap, consisting of a type tag, the size of the object, a pointer to the dispatch table (see below) of the class, and the values of the attributes of the class.

+ +

Objects are allocated by the alloc function in the execution environment, by copying a pointer to a prototype object. +The prototype object for a class is an object following the layout described above with attribute values set to their initial values in the corresponding class. +Thus, there is not need to perform the default initialization programmatically. +After copying the prototype object, the __init__ method is invoked to apply further custom initialization.

+ +

Dispatch Tables

+ +

Since all object instance of a class share the same set of methods, there is no need to copy the addresses of the corresponding functions into each object. +Instead, an object contains a pointer to a dispatch table, which contains the addresses (labels) of the functions implementing the methods. +Note that the methods of inherited method are also included in the dispatch table.

+ +

Compiling Methods

+ +

Methods are standard top-level functions and can be compiled using the same approach as used for those. +The first argument of a function is a pointer to the object to which the method is applied. +Since methods and attributes are accessed through this self argument, no special measures are needed.

+ +

Calling Methods

+ +

Method calls are similar to top-level function calles, but the first argument is passed as a receiver object. +That is, e0.f(e1, ...) is essentially a function call f(e0, e1, ...). +However, the actual function to be called should be obtained through the dispatch table to which the value of e0 points. +In order to access that object it should not be None.

+ +

Accessing Attributes

+ +

An attribute access e.x is implemented by accessing the object at the offset corresponding to the attribute x. +Also for attribute accesses, the object should not be None.

+ +

Compiling Lists

+ +

ChocoPy has lists, which are mutable sequences of a fixed length. So, lists behave like arrays in C. Operations on lists are length len, indexing lst[i], and concatenation lst1 + lst2. +Lists are represented as objects with len attributes to represent the elements of the list. +Concatenation constructs a new list copying the original lists.

+ +

Challenges

+ +

Garbage Collection

+ +

Objects are allocated on the heap. What happens when your program runs out of heap memory to allocate objects? Is compiled code memory safe? Develop a garbage collection algorithm for the execution environment of ChocoPy progams in order to support automatic memory management.

+ + + +

Peephole Optimization

+ +

A peephole optimization can be applied to generated code, i.e. to the AST of the generated program. A peephole optimization recognizes a couple of adjacent instructions and replaces them with something more efficient. +Extend your compiler with a peephole optimization phase after code generation that optimizes generated code. +Make sure that optimizations do preserve the semantics of the code.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/14.html b/lab/14.html new file mode 100644 index 0000000..7d2171f --- /dev/null +++ b/lab/14.html @@ -0,0 +1,259 @@ + + + + + + + + + Lab 14: Implementing Lists & Testing Your Compiler + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 14: Implementing Lists & Testing Your Compiler + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ December 24, 2021 +
+
+
+
+ +
+
+

Implement lists in ChocoPy.

+ +

Test all corner cases of the ChocoPy language and make sure your compiler deals with them correctly.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/15.html b/lab/15.html new file mode 100644 index 0000000..77a2ce3 --- /dev/null +++ b/lab/15.html @@ -0,0 +1,252 @@ + + + + + + + + + Lab 15: Finishing Touches + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 15: Finishing Touches + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ January 14, 2022 +
+
+
+
+ +
+
+

Make sure your code is ready for final submission.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/1a.html b/lab/1a.html new file mode 100644 index 0000000..433da25 --- /dev/null +++ b/lab/1a.html @@ -0,0 +1,959 @@ + + + + + + + + + Lab 1(a): Setting up Spoofax + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lab 1(a): Setting up Spoofax + +

+
+
+ + +
+
+ + + Project Set-up + + + + + + + +
+
+ September 07, 2021 +
+
+
+
+ +
+
+

For the course project and for some of the homework assignments we will use the Spoofax Language Workbench version Spoofax 3.

+ +

Installing Spoofax

+ +

Follow the tutorial on installing Spoofax3.

+ +

Language Projects

+ +

In Spoofax you develop a programming language in a ‘language project’.

+ +

Follow the tutorial on Creating a language project to create a project with syntax and syntax tests.

+ +

ChocoPy Project

+ +

Create a Spoofax project for the ChocoPy language. Configure your project to use the Chocopy name and cpy identifier, and place it in the root of your repository. See the image below for an example.

+ +

+ +

Your project can be located in either student-<yournetid>/chocopy.syntax or student-<yournetid>/chocopy. If you place your project in a nested folder, or in a differently named folder, the grader will automatically reject your submission.

+ +

Updating Spoofax

+ +

During the course of the project, bugfixes and new features may be added to Spoofax 3. The grader will always test your project using the most recent version of Spoofax 3 that is backwards compatible with the initial project version (0.11.6).

+ +

To update your local installation to the newest version of Spoofax 3, launch Eclipse and select Help from the top menu, followed by Install new software.

+ +

+ +

In the dialog that opens, enter the release URL of the latest Spoofax 3 release in the Work with: field. This URL starts with https://artifacts.metaborg.org and can be found on the Spoofax 3 releases page. After entering the URL, hit enter to load all available packages for that release, then select the Spoofax package.

+ +

+ +

Finally, hit next, accept the license if needed, and confirm the install. During installation, Eclipse may prompt that the package is unsigned. When this happens, select Install anyway.

+ +

+ +

Then, restart Eclipse. +After Eclipse has restarted, clean your ChocoPy language project by selecting it in the Package Explorer, choosing Project -> Clean from the main menu, and by pressing Clean. +After the project is cleaned, build it once and continue.

+ +

References

+ + + + +
    + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + + (editor). +
    + +
    + + + + + https://www.spoofax.dev/spoofax-pie/develop/ 2021 + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + + (editor). +
    + +
    + + + + + https://www.spoofax.dev/ 2021 + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/1b.html b/lab/1b.html new file mode 100644 index 0000000..1f0a04d --- /dev/null +++ b/lab/1b.html @@ -0,0 +1,975 @@ + + + + + + + + + Lab 1(b): Creating Syntax Tests + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lab 1(b): Creating Syntax Tests + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 07, 2021 +
+
+
+
+ +
+
+

Before getting started with your syntax definition, you set up a test suite for syntax analysis. +Develop the test suite in tandem with the development of your syntax definition. +The test suite consists of positive and negative test cases. +We will not grade this test suite, but you should develop one to get confidence in the quality of your syntax definition. +When you are asking for help from the course staff, we will first ask what tests you have written to demonstrate the problem.

+ +

Objectives

+ +

Develop a test suite for syntax analysis. +The test suite should provide

+ +
    +
  1. Syntactically valid and invalid test cases for the Start sort
    2. +
  2. Disambiguation tests for associativity and precedence in expressions.
    3. +
  3. Test cases for layout-sensitive language constructs
    4. +
  4. Test cases for mandatory and optional whitespace.
    5. +
+ +

Anatomy of a Test Suite

+ +

In Spoofax, a test suite consists of one or more .spt files. +Consider the following example file:

+ +
module example
+
+test FUN composite  [[func(1+1, 3*2)]] parse succeeds
+test FUN only comma [[func(,)]]        parse fails
+
+ +

The first line specifies the name of the test module, +In this example module, we test syntax of the ChocoPy language.

+ +

The last two lines specify a positive and a negative test case. +Each test case consists of + a name, + a code fragment in double square brackets, and + a condition which determines + what kind of test should be performed (parsing) and + what the expected outcome is (success or failure).

+ +

You get instant feedback while editing a test suite. +Since you have not yet implemented the ChocoPy editor, all your positive test cases fail. +You will work on this during the next lab. +To read more on SPT, visit the documentation.

+ +

ChocoPy Syntax Definition

+ +

To write your own test cases, you need to understand ChocoPy’s syntax. +You can find the definitive ChocoPy syntax definition in the Reference Manual.

+ +

Test Cases

+ +

You can now start writing your own test cases.

+ +

Lexical and Context-free Syntax

+ +

Start with the lexical syntax and think about valid and invalid identifiers. +Try to focus on corner cases such as

+ +
    +
  • identifiers made of a single character,
    • +
  • mixings of letters and digits (Which kinds of mixing are allowed? Which are not?),
    • +
  • underscores in identifiers.
    • +
+ +

The content for the tests for lexical symbols should not be surrounded by layout.

+ +

Next, you should walk through the whole syntax definition. +Keep thinking of valid and invalid code fragments. +Identify interesting corner cases and turn them into positive and negative test cases.

+ +

Try to keep your test cases small. +Each test case should focus on a single aspect. +For example, a negative test case should only contain one error. +The name of a test case should reflect the aspect it addresses.

+ +

You can organise test cases which focus on similar aspects in test modules. +Again, the name of a test module should reflect these aspects. +Finally, you can organise test suites in folders.

+ +

When writing tests that target the differences between Python and ChocoPy, only consider the syntactic differences. +For example, identifiers in Python are limited to a certain size. +This should not be checked by your tests as this limitation is not specified in the context-free grammar of Python.

+ +

Disambiguation

+ +

Next, you need to focus on disambiguation. +Come up with positive test cases which specify the correct ASTs for nested expressions. +As an alternative, you can also specify two fragments in concrete syntax, which should result in the same AST:

+ +
test left associative addition [[21 + 14 + 14]] parse to [[(21 + 14) + 14]]
+
+ +

You can find a table listing the associativity and priorities of operators in the reference manual. +Do not focus only on binary expressions.

+ +

You do not have to test for constructs that are non-associative. In lab 2, non-assoc leads to semantic errors rather than syntactic errors during parse time. Since our language does not have analysis at that point, we cannot test the semantic errors and thus they are also not required for this lab.

+ +

Layout

+ +

Finally, you can focus on layout. +Think about places where whitespace is not optional but mandatory and define corresponding positive and negative test cases. +Finish with test cases for single-line comments. +Single-line comments cannot only end with a newline character, but also at the end of file.

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Lennart C. L. Kats, Rob Vermaas, Eelco Visser. +
    + +
    + + + OOPSLA 2011 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + The reliability of compilers, interpreters, and development environments for programming languages is essential for effective software development and maintenance. They are often tested only as an afterthought. Languages with a smaller scope, such as domain-specific languages, often remain untested. General-purpose testing techniques and test case generation methods fall short in providing a low-threshold solution for test-driven language development. In this paper we introduce the notion of a language-parametric testing language (LPTL) that provides a reusable, generic basis for declaratively specifying language definition tests. We integrate the syntax, semantics, and editor services of a language under test into the LPTL for writing test inputs. This paper describes the design of an LPTL and the tool support provided for it, shows use cases using examples, and describes our implementation in the form of the Spoofax testing language. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/1c.html b/lab/1c.html new file mode 100644 index 0000000..84089bd --- /dev/null +++ b/lab/1c.html @@ -0,0 +1,485 @@ + + + + + + + + + Lab 1(c): Syntax Definition + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lab 1(c): Syntax Definition + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 07, 2021 +
+
+
+
+ +
+
+

In this lab, you define the concrete and abstract syntax of ChocoPy in Spoofax. +From this definition, you generate an Eclipse editor, which provides syntax checking, error recovery, and syntax highlighting.

+ +

In this lab, focus on translating the grammar in Figure 3 of the ChocoPy Reference Manual to an idiomatic context-free syntax definition with constructors. +In the next lab, you extend this definition with disambiguation rules for associativity, priority, and layout constraints.

+ +

Objectives

+ +

Develop a syntax definition for ChocoPy in SDF3 and generate an Eclipse editor from it. +The definition should include:

+ +
    +
  1. The start symbol Start
    2. +
  2. A lexical syntax definition for all ChocoPy lexemes
    3. +
  3. A context-free syntax definition ChocoPy complying with the grammar in Figure 3 of the reference manual
    4. +
  4. Constructor names for context-free syntax productions
    5. +
+ +

Syntax Definition

+ +

You should define your syntax in SDF3. +You can start by opening the file src/start.sdf3 in your chocopy project. +You can also split your syntax definition over several modules in src/ and import modules into module start.

+ +
module start
+imports lex
+imports expressions
+imports statements
+// etc.
+context-free start symbols Start
+
+ + + + + +

For more information on how to write SDF3 syntax definitions, you can also review the old SDF3 reference manual and the new SDF3 reference.

+ +

Note that both of these are written for Spoofax 2, and that not all information may still be correct in Spoofax 3!

+ +

Context-free Syntax

+ +

Start with the context-free syntax of the language. +Use the context-free grammar in the ChocoPy Language Reference Manual as a reference.

+ + + +

We recommend to use template productions for your context-free syntax definition, since they help when generating artifacts other than just the parser. +When you use template productions, you need to make sure that templates are correctly tokenised. +Otherwise, the parser would reject layout in certain positions, for example between [ and ] in an array type. +Check the SDF3 documentation for details. +In case you want to use < or > as symbols inside a template, you can use alternate template brackets [...].

+ +

Lexical Syntax

+ +

Continue with the lexical syntax definition including identifiers, integer, and simple layout. + +

+ +

First, define lexical syntax rules:

+ +
lexical syntax
+  ID     = ...
+  INT    = ...
+  LAYOUT = ...
+
+ +

Second, define follow restrictions to ensure longest match:

+ +
lexical restrictions
+  ID -/- ...
+  INT -/- ...
+context-free restrictions
+ LAYOUT? -/- ...
+
+ +

Finally, use rejection rules to rule out reserved words.

+ +
lexical syntax
+  ID = ... {reject}
+
+ +

Testing Your Syntax Definition

+ +

You now can check your tests.

+ +

It is important to get your grammar working at this stage. Do not move on if you have issues here, since there is a high chance that these issues influence your other tests as well. If you experience weird behaviour on your tests or a large amount of ambiguity, this is most likely caused by an erroneous definition of LAYOUT.

+ +

Finally, you should add lexical syntax rules for comments to your syntax definition. +Start with single line comments. +Do not forget to define follow restrictions.

+ + + + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/1d.html b/lab/1d.html new file mode 100644 index 0000000..0f4ab11 --- /dev/null +++ b/lab/1d.html @@ -0,0 +1,555 @@ + + + + + + + + + Lab 1(d): Git(Lab), CI, and Submissions + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 1(d): Git(Lab), CI, and Submissions + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 07, 2021 +
+
+
+
+ +
+
+

We use the Git version control system with gitlab.ewi.tudelft.nl to manage submissions and grade assignments.

+ +

Git

+ +

Download and install Git from this page. We will be using the Git command-line, since it is more powerful and easier to troubleshoot than GUI clients. On Linux and macOS, installing Git should provide a git command on your shell. On Windows, Git installs Git BASH to provide a command-line shell, which you need to start to use Git from the command-line.

+ +

We will explain the steps needed to work on and submit assignments on this page. To learn the basics of git, read git - the simple guide and try out the Git command-line. If you’d like to learn more, have a look at these resources.

+ +

Repository Structure

+ +

Let’s look at the repository structure first.

+ +

On GitLab, we will create a private git repository for you, which is owned by us, and is only visible to you and the compiler construction team. This repository will host assignment templates and your submissions in protected branches. Note that you do not have write access to protected branches. You can only read from it and submit assignments to it by opening merge requests. This is to ensure that you cannot mess with submitted assignments after the deadline. You do have access to non-protected branches, and this is where you can push your work frequently.

+ +

Getting started

+ +

First, find your private repository in the CS4200/2021-2022 group on GitLab, it should be called student-id where id is your NetID.

+ +

To actually do work in the Git repository, you need to make a local clone of the repository on your computer. You can find the URL needed to clone on the bottom right of the page, be sure to use HTTPS.

+ +

Clone URL

+ +

Open up the command line and make a local clone with your URL:

+ +
git clone https://gitlab.ewi.tudelft.nl/CS4200/2021-2022/student-netid.git
+
+ +

Now cd into the local clone and confirm that it works:

+ +
cd student-netid
+git status
+
+ +

which should output something like:

+ +
On branch master
+Your branch is up-to-date with 'origin/master'.
+nothing to commit, working directory clean
+
+ +

Your local repository is set up now! Follow the steps below to work on an assignment.

+ +

Starting an assignment

+ +

You work on each assignment in its own development branch, named milestone-0-develop, milestone-1-develop, etc. +The correct assignment branch must be checked out in your local Git repository to be able to work on it. +The steps to check out a branch depend on whether we provide you with a template, or if you continue with work from a previous assignment.

+ +

Template

+ +

If the assignments asks you to check out a template, which for example milestone-0-template does, use the following steps:

+ +
git fetch origin
+git checkout -b milestone-0-develop origin/milestone-0-template
+git push -u origin milestone-0-develop
+
+ +

This checks out a fresh branch from the remote repository. It does not contain any of your previous work.

+ +

Continue from previous assignment

+ +

If the assignment asks you to continue from the previous assignment, which for example milestone 2 does, use the following steps instead:

+ +
git checkout -b milestone-2-develop
+git push -u origin milestone-2-develop
+
+ +

The new milestone-2-develop branch will be in an identical state to the milestone-1-develop branch, but changes will only be committed to the milestone-2-develop branch, leaving the milestone-1-develop branch as is. +Now you have the assignment branch checked out in your local repository and can start working.

+ +

Saving work

+ +

Whenever you have changes that you’d like to save, such as after getting (a part of) the assignment working, you need to add, commit, and push your changes:

+ +
git add --all
+git commit -m "Message describing your changes"
+git push
+
+ +

Continuous Integration (CI)

+ +

The template contains a .gitlab-ci.yml file that configures continuous integration in GitLab. +It runs gradle buildAll on each push to milestone-*-develop. It also invokes the grader when the project is submitted using a merge request.

+ +

By default, both language projects used for milestones 0-2 are enabled in the gradle build. If, for some reason, building of a project needs to be disabled, the following lines should be commented out in the settings.gradle.kts file.

+ +
includeLanguageIfExists("chocopy.syntax")
+includeLanguageIfExists("chocopy.types")
+
+ +

Enabling the build of your language projects in CI is recommended but not required when submitting your work.

+ +

Submitting a milestone

+ +

Merge requests are used to submit a milestone from your develop branch to its corresponding submission branch.

+ +

Make sure you’ve pushed all your changes first, then go to your repository on GitLab. +Navigate to Merge Requests and click the New merge request button:

+ +

New merge request

+ +

Select the develop branch as source and its corresponding submission branch as target. +Click Compare branches and continue.

+ +

If all is well, you can review your changes and you can press the Submit merge request button to submit your milestone. If not, check the troubleshooting section.

+ +

Merge request

+ +

We will grade your milestone and post the results as an issue in your project, so check back later.

+ +

Switching to another milestone

+ +

If you’d like to work on an another milestone, for example to fix things for a new submission, you can just check out the branch for that milestone. +Be sure to push any changes to your current branch first, then check out the branch for the milestone you wish to switch to:

+ +
git checkout milestone-1-develop
+
+ +

Pulling in changes from template

+ +

If there’s something wrong in the template for a milestone, we fix it in the template branch, and you have to merge in those changes. +Use the following commands to merge in changes (use the correct branch!):

+ +
git fetch origin
+git merge origin/milestone-1-template
+git push
+
+ +

In most cases, Git will automatically merge in any changes, but sometimes conflicts can occur. See Resolving a merge conflict on how to resolve conflicts.

+ +

Git GUI clients

+ +

This guide uses command-line Git commands, but if you’d rather use a GUI, use SourceTree.

+ +

Troubleshooting

+ +

Cannot push

+ +

No access/rights

+ +

When Git complains about not being able to push because you do not have access or rights to the repository, this probably means that you’re trying to push to a template or submission branch rather than a develop branch.

+ +

Out of date branch

+ +

You cannot push changes to a remote when that remote has changes that you haven’t yet pulled, you’ll get an error like:

+ +
git push
+To ...
+ ! [rejected]        milestone-1-develop -> milestone-1-develop (non-fast-forward)
+error: failed to push some refs to '...'
+hint: Updates were rejected because the tip of your current branch is behind
+hint: its remote counterpart. Integrate the remote changes (e.g.
+hint: 'git pull ...') before pushing again.
+hint: See the 'Note about fast-forwards' in 'git push --help' for details.
+
+ +

First pull changes with git pull and then push your changes.

+ +

Cannot pull

+ +

Local changes

+ +

When you have changes in your local repository that you have not committed yet, and you try to pull, Git may complain about your changes being overwritten. +First add and commit your changes locally with:

+ +
git add --all
+git commit -m "Message describing your changes"
+
+ +

and then pull changes with git pull.

+ +

Cannot automatically merge merge request

+ +

If a merge request cannot be automatically merged, your branch is out of date with the template branch. +Merge in changes from the template (use the correct branch!):

+ +
git fetch origin
+git merge origin/milestone-1-template
+git push
+
+ +

Resolving merge conflicts

+ +

See Resolving a merge conflict on how to resolve merge conflicts. +You can also try a GUI merge tool such as DiffMerge to resolve merge conflicts.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/2.html b/lab/2.html new file mode 100644 index 0000000..11027c0 --- /dev/null +++ b/lab/2.html @@ -0,0 +1,1060 @@ + + + + + + + + + Lab 2: Disambiguation + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 2: Disambiguation + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 14, 2021 +
+
+
+
+ +
+
+

In this lab you adapt and extend the syntax definition of the previous lab with disambiguation rules and layout constraints in order to ensure that the syntax definition is unambiguous.

+ +

Objectives

+ +

Adapt and extend the syntax definition that you developed in the previous lab such that it

+ +
    +
  1. is an idiomatic SDF3 definition
    2. +
  2. realizes disambiguation using declarative disambiguation rules
    3. +
  3. defines layout constraints to enforce layout-sensitive syntax
    4. +
+ +

Disambiguation Rules

+ +

Section 4.1 of the ChocoPy reference manual specifies the precedence and associativity rules of the language. +Translate this into associativity and priority rules in SDF3.

+ +

You need disambiguation constructs to disambiguate your syntax definition. +You can specify the associativity with attributes left, right, or non-assoc (See documentation). +These attributes are added to the end of a rule:

+ +
context-free syntax
+  Exp.Constr1 = ... {left}
+
+ +

You can specify precedence in a context-free priorities section. +In this section, you order groups of rules:

+ +
context-free priorities
+  {
+    Exp.Constr1
+    Exp.Constr2
+  } > { ... } > ...
+
+ +

You can also define the associativity of a group:

+ +
context-free priorities
+  { left:
+    Exp.Constr1
+    Exp.Constr2
+  } > { ... } > ...
+
+ +

Finally you should also consider specifying a bracket rule for disambiguation of expressions.

+ +

Layout Constraints

+ +

The syntax of ChocoPy is layout-sensitive, meaning that indentation and newlines are significant. +To enable layout-sensitive parsing in Spoofax 3, we need to make some changes to the spoofaxc.cfg file. +In the spoofaxc.cfg file, replace:

+ +
sdf3 {}
+
+ +

with:

+ +
sdf3 {
+  parse-table-generator {
+    layout-sensitive = true
+  }
+}
+
+ +

and replace:

+ +
parser {
+  default-start-symbol = sort Start
+}
+
+ +

with:

+ +
parser {
+  default-start-symbol = sort Start
+  variant = jsglr2 {
+    preset = LayoutSensitive
+  }
+}
+
+ +

In the ChocoPy grammar in Figure 3 in the reference manual, layout-sensitive parsing is defined using NEWLINE, INDENT, and DEDENT tokens. +Section 3.1 describes the meaning of these tokens.

+ +

Translate these tokens into layout constraints and layout declarations as presented in Lecture 3 and in the papers cited below.

+ +

Additional documentation on layout constraints can also be found in the Spoofax 2 documentation. Please note that this documentation is for Spoofax 2, and may therefore not fully correspond to Spoofax 3 behavior.

+ +

CExpr and Expr

+ +

The ChocoPy reference manual uses two separate expression sorts in its grammar, in order to keep compatibility with Python and to disambiguate. SDF3 has disambiguation features that makes this separation of sorts unneccessary. As a result, we expect you to implement a grammar that only uses a single expression sort. For more information on grading, see milestone 1.

+ +

One particularity is that ChocoPy does not allow any binary operators that feature expressions such as negation on the right-hand side. Indeed, True == not False is invalid syntax in ChocoPy. In order to emulate this restriction using a flattened grammar in SDF3, you can use the following priority directive syntax:

+ +
context-free priorities
+  A.CA <n> .> B.CB
+
+ +

This restriction states that the n-th (0-indexed) child of A.CA cannot be a term of type B.CB. For example:

+ +
context-free grammar
+  Expr.Eq = <<Expr> == <Expr>>
+  Expr.Not = <not <Expr>>
+
+context-free priorities
+  Expr.Eq <2> .> Expr.Not
+
+ +

This will reject a not expression from appearing on the right-hand side of an equality expression.

+ +

Using this syntax and other SDF3 disambiguation features discussed in the lectures, you should be able to implement a ChocoPy grammar in SDF3 that adheres to the reference manual while only using a single sort for expressions.

+ +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Luis Eduardo de Souza Amorim, Michael J. Steindorfer, Sebastian Erdweg, Eelco Visser. +
    + +
    + + + SLE 2018 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + In layout-sensitive languages, the indentation of an expression or statement can influence how a program is parsed. While some of these languages (e.g., Haskell and Python) have been widely adopted, there is little support for software language engineers in building tools for layout-sensitive languages. As a result, parsers, pretty-printers, program analyses, and refactoring tools often need to be handwritten, which decreases the maintainability and extensibility of these tools. Even state-of-the-art language workbenches have little support for layout-sensitive languages, restricting the development and prototyping of such languages. In this paper, we introduce a novel approach to declarative specification of layout-sensitive languages using layout declarations. Layout declarations are high-level specifications of indentation rules that abstract from low-level technicalities. We show how to derive an efficient layout-sensitive generalized parser and a corresponding pretty-printer automatically from a language specification with layout declarations. We validate our approach in a case-study using a syntax definition for the Haskell programming language, investigating the performance of the generated parser and the correctness of the generated pretty-printer against 22191 Haskell files. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Sebastian Erdweg, Tillmann Rendel, Christian Kästner, Klaus Ostermann. +
    + +
    + + + SLE 2012 + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Sebastian Erdweg, Felix Rieger, Tillmann Rendel, Klaus Ostermann. +
    + +
    + + + haskell 2012 + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/3a.html b/lab/3a.html new file mode 100644 index 0000000..b2e5462 --- /dev/null +++ b/lab/3a.html @@ -0,0 +1,895 @@ + + + + + + + + + Lab 3(a): Setting up Static Analysis + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lab 3(a): Setting up Static Analysis + +

+
+
+ + +
+
+ + + Project Set-up + + + + + + + +
+
+ September 22, 2021 +
+
+
+
+ +
+
+

In this lab, you define the static analysis of ChocoPy in Spoofax. +Follow the instructions from Lab 1(a) for using Spoofax.

+ +

Modifying the Static Analysis

+ +

Static analysis is implement in the Statix language of Spoofax. +The /src/main.stx file is the main Statix file. +You can define static analysis rules in this main file, but it is probably a good idea to distribute your Statix definition over multiple modules, as Statix is a modular language. +If you create new modules, you must import them into /src/main.stx.

+ +

References

+ + + + +
    + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + + (editor). +
    + +
    + + + + + https://www.spoofax.dev/spoofax-pie/develop/ 2021 + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + + (editor). +
    + +
    + + + + + https://www.spoofax.dev/ 2021 + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/3b.html b/lab/3b.html new file mode 100644 index 0000000..ca73102 --- /dev/null +++ b/lab/3b.html @@ -0,0 +1,615 @@ + + + + + + + + + Lab 3(b): Creating Static Analysis Tests + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 3(b): Creating Static Analysis Tests + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 22, 2021 +
+
+
+
+ +
+
+

Before getting started with your static analysis definition, you set up a test suite for static analysis. Develop the test suite in tandem with the development of your static analysis. The test suite consists of positive and negative test cases. We will not grade this test suite, but you should develop one to get confidence in the quality of your static analysis. When you are asking for help from the course staff, we will first ask what tests you have written to demonstrate the problem.

+ +

Objectives

+ +

In your chocopy project, develop a test suite for static analysis. The test suite should provide

+ +
    +
  1. Test cases for types
    2. +
  2. Test cases for name resolution
    3. +
  3. Test cases for errors
    4. +
+ +

Testing Type Constraints

+ +

See the example tests in the WebLab homework assignments for Week 3.

+ +

Testing Reference Resolution

+ +

In test cases for reference resolution, + you write syntactically correct programs and + mark names at definition and use sites with inner square bracket blocks. +You can then relate the use site with the definition site in a resolve x to y clause, + using numbers to refer to the inner blocks. +For example, the following two test cases require to resolve the type Foo to the name in the definition of class Foo:

+ +
module resolution
+
+test class name resolution[[
+class [[A]](object):
+	pass
+
+class B(object):
+	a:[[A]] = None
+]] resolve #2 to #1
+
+test class field resolution[[
+class Foo(object):
+	[[bar]]:int = 0
+
+foo:Foo = None
+foo = Foo()
+print(foo.[[bar]])
+]] resolve #2 to #1
+
+ +

After copying this into an SPT file Spoofax will add the error “Reference resolution failed” and “No constraint generation rule for …“. This is expected, since your project is missing an implementation for reference resolution (this is part of the next lab).

+ +

You can use fixtures to avoid repeating parts in similar test cases. See the SPT documentation for details.

+ +

You should come up with test cases for the resolution of class names, field names, parameter names, and variable names. +Start with simple test cases, but keep in mind that coverage is the main criterion for your grade. +It is important to think about + forward and backward references, + global and non-local declarations in functions, + and resolution in the presence of homonyms.

+ +

Make sure that there are no errors in tests with a resolve x to y clause, these tests are invalid when there are errors.

+ +

Testing Error Checking

+ +

In test cases for error checking, you need to specify the number of errors, warnings, or notes in a +test case in errors, warnings, or notes clauses. For example, the following test cases +specify a correct ChocoPy program, a program with two errors which are reported on the name of a +duplicate class Foo, and another program with an error which is reported on the name of an unknown +class Bar:

+ +
module correctness
+
+language chocopy
+
+test correct program [[
+class Counter(object):
+	count:int = 0
+
+	def getCount(self:Counter)->int:
+		return self.count
+
+	def increaseCount(self:Counter):
+		self.count = self.count + 1
+]] 0 errors
+
+test incorrect program [[
+class Counter(object):
+	count:bool = False
+
+	def getCount(self:Counter)->int:
+		return self.count
+
+	def increaseCount(self:Counter):
+		self.count = self.count + 1
+]] >= 1 errors // or 3 errors
+
+test error on unknown class [[
+class Foo(object):
+	bar:Bar = None
+]] >= 1 errors
+
+ +

You can start with test cases for duplicate and missing definitions. Similar to your syntax test +cases, you can pair up positive (0 errors) and negative test cases. For duplicate definitions, we +expect errors on the definitions with the same name.

+ +

The number of errors can be hard to predict, because errors sometimes cascade. Therefore, if you +expect any errors, you should use the >= 1 errors expectation, even if you expect a specific +number of errors. For example, this expectation was used in the duplicate class test, even though we +would expect exactly two errors.

+ +

Next, you should develop test cases for fields and variables which hide global variables, global and non-local declarations, and class +instantiation, subclassing, referencing. Again, you should keep in mind that coverage is the main +criterion for your grade.

+ +

Testing Types of Expressions

+ +

In test cases for type analysis, + you write syntactically correct programs and + mark expressions with inner square bracket blocks. +You can then specify the expected type of the marked expression in a run x to y clause. +For example, the following two test cases require an integer literal to be of type Int() +and a variable reference to be of its declared type Bool():

+ +
module types
+
+test integer literal [[
+print([[1]])
+]] run get-type on #1 to Int()
+
+test boolean condition [[
+b1:bool = True
+b2:bool = False
+if [[b1 and b2]]:
+	print("Yes!")
+]] run get-type	on #1 to Bool()
+
+ +

In order for these tests to succeed, you need to define your own Stratego rule get-type. For this, you can copy and paste the following code into src/main.str2

+ +
signature
+  sorts
+    Type
+
+  constructors
+    Int : Type
+    Bool : Type
+    String : Type
+    ClassType : scope * ID -> Type
+    List : Type -> Type
+    NoneType : Type
+    EmptyList : Type
+    Object : Type
+    FunType : Type * list(Type) -> Type
+    // We expect to get these types in the grading pipeline
+    // you are free to use your own (custom) types in your Statix file
+    // as long as you write transformation rules from your types to our types
+    // in resolve-type
+
+    // Here you can define the signature of the types you have used in your Statix definitions.
+    // This is just an example on how to do it.
+    // signature
+    //   sorts
+    //     MyCustomType
+    //
+    //  constructors
+    //    MyCustomIntType : MyCustomType
+    //    MyCustomClassType : ID * scope -> MyCustomType
+
+rules
+  // get-type: SimpleStatement(expr) -> <get-type> expr // An example on how to match on AST nodes.
+  get-type: expr -> type' // Defines a rule which matches on node and returns type'
+    where
+      // Assigns variable a to be the result of the Statix analysis of the entire program (or throws an error)
+      a := <stx-get-ast-analysis <+ fail-msg(|$[no analysis on node [<strip-annos;write-to-string> expr]])>;
+      // Gets the type of the given node (or throws an error)
+      type := <stx-get-ast-type(|a) <+ fail-msg(|$[no type on node [<strip-annos;write-to-string> expr]])> expr;
+      // Calls a rule to convert the type given by Statix to a type for our tests.
+      type' := <resolve-type> type
+
+  // resolve-type: MyCustomIntType() -> Int() // This rule matches on your custom type, and returns a type we need in our tests.
+  // resolve-type: MyCustomClassType(name, scope) -> ClassType(scope, name) // This rule matches on your custom type, and returns a type we need in our tests.
+  // TODO: Define your own transformation rules to transform types from your Statix defintion to the types we expect in our tests
+  resolve-type: a -> a // This rule matches on a and returns a (trivial rule given as example). This implies no transformation actually takes place. Define your own rules ABOVE this rule.
+
+  fail-msg(|msg) = err-msg(|$[get-type: [msg]]); fail
+
+ +

This code defines a Stratego rule get-type which we can use to map AST Nodes to their types using the results of the Statix analysis. In our tests for the Early Feedback and Grading, we expect certain types, so it is highly recommended to use the same types in your Statix rules. If not, you might need to perform some extra transformations from your types to our types in the resolve-type rule.

+ +

For more information on the Stratego language, visit the Stratego documentation.

+ +

You can use fixtures to avoid repeating parts in similar test cases. See the +SPT documentation +for details.

+ +

When applying get-type to objects, we expect a ClassType constructor.

+
test class type [[
+class Foo(object):
+	pass
+[[Foo()]]
+]] run get-type on #1 to ClassType(_, "Foo") // We don't care about the scope, only the classname.
+
+ +

You should come up with test cases for the types of all kinds of expressions. Just like previous +testing assignments, this assignment is all about the coverage of your test suite.

+ +

The constructors for various types are:

+ +
    +
  • Integer: Int()
    • +
  • Boolean: Bool()
    • +
  • String: String()
    • +
  • List of Type T: List(T)
    • +
  • Empty list type: EmptyList()
    • +
  • None type: NoneType()
    • +
  • Class with name Foo and scope s: ClassType(s, "Foo")
    • +
  • Function with return Type RT and a list of parameter types PTs: FunType(RT, PTs)
    • +
  • Object: Object()
    • +
+ +

Tests that contain a run x to y clause are invalid if the semantic analysis yields errors. Ensure that the snippet of ChocoPy that you are testing does not contain any errors if you want to use get-type or other strategies.

+ +

If you use a start symbol other than Program, replace any samples in this guide that use Program with your start symbol instead.

+ +

Make sure to actually append your AST nodes with their types using @x.type := T (in a similar way to assigning the ref property, which is needed for name resolution). Otherwise the rule will not find any types on your AST node.

+ +

Testing Method Name Resolution

+ +

Consider the following test case as an example:

+ +
test method name resolution [[
+class Foo(object):
+	def [[run]](self:Foo)->int:
+		return 1
+Foo().[[run]]()
+]] resolve #2 to #1
+
+ +

The type of the callee expression determines the class in which the method declaration can be found. +In this example, the expression Foo() is of type ClassType(_, "Foo") and +the corresponding class Foo contains a method declaration for run().

+ +

You should come up with test cases for the resolution of method names. +Start with simple test cases, but keep in mind that method name resolution is quite complex + and that coverage is the main criterion for your grade. +It is important to think about forward and backward references, + resolution in the presence of homonyms and overriding, + and the influence of class hierarchies on resolution.

+ +

You should also come up with test cases for error checking on method names. +This should include test cases for errors on duplicate definitions, missing definitions, and method overloading. +Similar to previous test cases, you can pair up positive (0 errors) and negative test cases.

+ +

Make sure that there are no errors in tests with a resolve x to y clause. These tests are invalid when there are errors.

+ +

Testing Type Error Checking

+ +

A type error occurs, when the type of an expression does not conform to its expected type. +Consider the following test case as an example:

+ +
test print boolean [[
+def printInt(i:int):
+	print(i)
+printInt(True)
+]] 1 error
+
+ +

Type errors can occur in statements, expressions, and method declarations. +You should come up with test cases for such errors. +Subtyping is a common source for errors not only in programs, but also in language implementations. +It is therefore important to have positive and negative typing tests, which involve correct and incorrect subtyping.

+ +

Again, keep in mind that coverage is the main criterion for your grade.

+ +

Number of errors

+ +

Similar to the previous testing lab, you need to be careful about the number of errors, because +errors sometimes cascade. For example, if you expect 2 errors, you should use the >= 2 errors +expectation, even if you expect an exact number of errors.

+ +

A final note

+

A great way of testing expected ChocoPy behavior, when you want to know whether something is allowed or not, is to try it out on the ChocoPy Website

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/3c.html b/lab/3c.html new file mode 100644 index 0000000..78ab78a --- /dev/null +++ b/lab/3c.html @@ -0,0 +1,332 @@ + + + + + + + + + Lab 3(c): Simple Types + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 3(c): Simple Types + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 22, 2021 +
+
+
+
+ +
+
+

Read Section 4 of the ChocoPy Reference Manual with the type rules of the language. +In the remaining labs of the course, you will translate those type rules into formal Statix rules in order to implement a type checker. +You should submit the result, a complete type system implementation of ChocoPy in Statix, in Milestone 2. +You can proceed at your own pace, but in this and the following lab descriptions, we give some ideas for ordering the task.

+ +

In this lab, follow Lecture 4 and the homework assignments for Week 3 to define a predicate (say typeOfExp) to test the type correctness of expressions without names (variables, functions).

+ +

For more information on Statix, see also the Spoofax 2 documentation on Statix. Most of the information in this document will also apply to Spoofax 3.

+ +

For now, ignore the contexts in the typing rules. You can see (by tracing the rules) that for expressions not involving names, those contexts are irrelevant. In the predicate, you can either leave the context out entirely, or you can pass around a dummy ‘scope’. (We will study scopes in the next lecture.) +In the latter case, you need to figure out how to create the dummy scope. In the former case you will need to add scopes later, but that should be a simple find-and-replace action.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/4.html b/lab/4.html new file mode 100644 index 0000000..d6525a6 --- /dev/null +++ b/lab/4.html @@ -0,0 +1,318 @@ + + + + + + + + + Lab 4: Local Names + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 4: Local Names + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 29, 2021 +
+
+
+
+ +
+
+

In this lab, follow Lecture 5 and the homework assignments for Week 4 to extend the type checking predicates with name resolution.

+ +

Continue your study of Section 4 of the ChocoPy Reference Manual with the type rules of the language.

+ +

Start with the definition of local variable declarations and variable references in this first lab. Next add formal parameters of functions.

+ +

Define test cases to validate your definition. Make sure to test for shadowing of variables. Did you implement the correct visibility policy in your resolution queries?

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/5.html b/lab/5.html new file mode 100644 index 0000000..e88f93b --- /dev/null +++ b/lab/5.html @@ -0,0 +1,311 @@ + + + + + + + + + Lab 5: Functions and Classes + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 5: Functions and Classes + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ October 06, 2021 +
+
+
+
+ +
+
+

Continue your study of Section 4 of the ChocoPy Reference Manual with the type rules of the language.

+ +

Finish the definition of local variables and function parameters (Lab 4).

+ +

Move on to definition of function definitions and function calls, classes with inheritance, method definitions, and method calls.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/6.html b/lab/6.html new file mode 100644 index 0000000..feb598d --- /dev/null +++ b/lab/6.html @@ -0,0 +1,304 @@ + + + + + + + + + Lab 6: Preparing Milestone 2 Submission (2020) + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 6: Preparing Milestone 2 Submission (2020) + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ October 13, 2021 +
+
+
+
+ +
+
+

You have two labs left until the Milestone 2.

+ +

Make sure to test the corner cases of type checking ChocoPy and use the early feedback to see how well you are doing.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/7.html b/lab/7.html new file mode 100644 index 0000000..99088b0 --- /dev/null +++ b/lab/7.html @@ -0,0 +1,299 @@ + + + + + + + + + Lab 7: Finish Front-End (2020) + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 7: Finish Front-End (2020) + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ October 20, 2021 +
+
+
+
+ +
+
+

You have one lab left until the Milestone 2.

+ +

Make sure to test the corner cases of type checking ChocoPy and use the early feedback to see how well you are doing.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/8.html b/lab/8.html new file mode 100644 index 0000000..08736e6 --- /dev/null +++ b/lab/8.html @@ -0,0 +1,659 @@ + + + + + + + + + Lab 8: Simple Transformation + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 8: Simple Transformation + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ November 12, 2021 +
+
+
+
+ +
+
+

During this lab get to know Stratego by making the homework assignments for Week 1 either in Spoofax 3 or in WebLab.

+ + + + + + + + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lab/9.html b/lab/9.html new file mode 100644 index 0000000..520fb36 --- /dev/null +++ b/lab/9.html @@ -0,0 +1,550 @@ + + + + + + + + + Lab 9: A Basic Compiler Pipeline + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lab 9: A Basic Compiler Pipeline + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ November 19, 2021 +
+
+
+
+ +
+
+

In this lab you make a start with compiling ChocoPy to RISC-V code using Stratego. +You translate expression statements with expressions containing constants, operators, and variables. +While it would be possible to completely compute the values of such expressions using constant folding, this is not the aim of this lab. +Rather, we want the basis for a translation of expressions using variables and function calls as well.

+ +

A Minimial Viable Example

+ +

The following is a minimal ChocoPy program:

+ +
y : int = 3
+1 + y
+
+ +

Aim to get the complete compiler pipeline working for this example first. +Then work on extending the various stages.

+ +

Nano-Pass Compiler Architecture

+ +

Follow the setup of the pipeline in Essentials of Compiling consisting of the following strategies:

+ +
compile =
+  compile-cpy-to-cir
+  ; compile-cir-to-rv32im
+  ; compile-rv32im
+
+compile-cpy-to-cir =
+  explicate-types
+  ; desugar
+  ; uniquify
+  ; remove-complex-operands
+  ; explicate-control
+
+compile-cir-to-rv32im =
+  select-instructions-cprogram
+
+compile-rv32im =
+  assign-homes
+  ; patch-instructions
+
+ +

Be sure to replace the compile rule in src/compile.str2 with the above compile strategy.

+ +

RISC-V Resources

+ +

Consult the following resources as documentation on the RISC-V architecture and instruction set:

+ + + +

Prebuilt

+ +

The prebuilt directory of your project template should have

+ +
    +
  • The ChocoPy signature and pretty-printer
    • +
  • The RV32IM signature and pretty-printer
    • +
+ +

The template should also have implementations of the ChocoPy parser and type checker.

+ +

Compiler Testing

+ +

Develop tests in SPT to test your compiler.

+ +
module test
+
+language chocopy
+
+test 1+1 [[
+1+1
+]] transform "To RV32IM and execute" to "2"
+
+ + + + + + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/1.html b/lecture/1.html new file mode 100644 index 0000000..0f518c5 --- /dev/null +++ b/lecture/1.html @@ -0,0 +1,957 @@ + + + + + + + + + Lecture 1: What is a Compiler? + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 1: What is a Compiler? + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ September 02, 2021 +
+
+
+
+ +
+
+

In the opening lecture we look at what compilers are, why you would want to study them, and how the course is organized.

+ +

Slides

+ + + +

Reading Material

+ +

The lecture discusses the traditional compiler pipeline. In the course we will be using the Spoofax language workbench to define and implement languages. The [OOPSLA 2010] paper describes the design of Spoofax. The [SEFM 2020] paper gives an overview of SDF3, the language we will for syntax definition.

+ +

Tutorial

+ + + +

Introduction to a tutorial on Spoofax at the PLDI 2020 conference in June 2020.

+ +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Luis Eduardo de Souza Amorim, Eelco Visser. +
    + +
    + + + SEFM 2020 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + SDF3 is a syntax definition formalism that extends plain context-free grammars with features such as constructor declarations, declarative disambiguation rules, character-level grammars, permissive syntax, layout constraints, formatting templates, placeholder syntax, and modular composition. These features support the multi-purpose interpretation of syntax definitions, including derivation of type schemas for abstract syntax tree representations, scannerless generalized parsing of the full class of context-free grammars, error recovery, layout-sensitive parsing, parenthesization and formatting, and syntactic completion. This paper gives a high level overview of SDF3 by means of examples and provides a guide to the literature for further details. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Rohan Padhye, Koushik Sen, Paul Hilfnger, Rohan Padhye, Koushik Sen, Paul Hilfnger. +
    + +
    + + + + + + + [pdf, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Lennart C. L. Kats, Eelco Visser. +
    + +
    + + + OOPSLA 2010 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Spoofax is a language workbench for efficient, agile development of textual domain-specific languages with state-of-the-art IDE support. Spoofax integrates language processing techniques for parser generation, meta-programming, and IDE development into a single environment. It uses concise, declarative specifications for languages and IDE services. In this paper we describe the architecture of Spoofax and introduce idioms for high-level specifications of language semantics using rewrite rules, showing how analyses can be reused for transformations, code generation, and editor services such as error marking, reference resolving, and content completion. The implementation of these services is supported by language-parametric editor service classes that can be dynamically loaded by the Eclipse IDE, allowing new languages to be developed and used side-by-side in the same Eclipse environment. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/10.html b/lecture/10.html new file mode 100644 index 0000000..1d459f0 --- /dev/null +++ b/lecture/10.html @@ -0,0 +1,1093 @@ + + + + + + + + + Lecture 10: Program Transformation by Term Rewriting + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 10: Program Transformation by Term Rewriting + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ November 11, 2021 +
+
+
+
+ +
+
+

In this lecture we start with the second part of the compiler course in which we will study the construction of compiler back-ends.

+ +

We start with the study of term rewrite rules to define program transformations. And we look at the use of programmable strategies in the Stratego transformations to control application of rewrite rules.

+ +

Slides

+ + + +

References

+ + + + +
    + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Essentials of Compilation. An Incremental Approach in Racket + +
    + +
    + Jeremy G. Siek. +
    + +
    + + + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Jeff Smits, Eelco Visser. +
    + +
    + + + SLE 2020 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + The Stratego language supports program transformation by means of term rewriting with programmable rewriting strategies. Stratego's traversal primitives support concise definition of generic tree traversals. Stratego is a dynamically typed language because its features cannot be captured fully by a static type system. While dynamic typing makes for a flexible programming model, it also leads to unintended type errors, code that is harder to maintain, and missed opportunities for optimization. In this paper, we introduce a gradual type system for Stratego that combines the flexibility of dynamically typed generic programming, where needed, with the safety of statically declared and enforced types, where possible. To make sure that statically typed code cannot go wrong, all access to statically typed code from dynamically typed code is protected by dynamic type checks (casts). The type system is backwards compatible such that types can be introduced incrementally to existing Stratego programs. We formally define a type system for Core Gradual Stratego, discuss its implementation in a new type checker for Stratego, and present an evaluation of its impact on Stratego programs. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Lennart C. L. Kats, Eelco Visser. +
    + +
    + + + OOPSLA 2010 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Spoofax is a language workbench for efficient, agile development of textual domain-specific languages with state-of-the-art IDE support. Spoofax integrates language processing techniques for parser generation, meta-programming, and IDE development into a single environment. It uses concise, declarative specifications for languages and IDE services. In this paper we describe the architecture of Spoofax and introduce idioms for high-level specifications of language semantics using rewrite rules, showing how analyses can be reused for transformations, code generation, and editor services such as error marking, reference resolving, and content completion. The implementation of these services is supported by language-parametric editor service classes that can be dynamically loaded by the Eclipse IDE, allowing new languages to be developed and used side-by-side in the same Eclipse environment. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Martin Bravenboer, Arthur van Dam, Karina Olmos, Eelco Visser. +
    + +
    + + FUIN 69(1-2) 2006 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + The applicability of term rewriting to program transformation is limited by the lack of control over rule application and by the context-free nature of rewrite rules. The first problem is addressed by languages supporting user-definable rewriting strategies. The second problem is addressed by the extension of rewriting strategies with scoped dynamic rewrite rules. Dynamic rules are defined at run-time and can access variables available from their definition context. Rules defined within a rule scope are automatically retracted at the end of that scope. In this paper, we explore the design space of dynamic rules, and their application to transformation problems. The technique is formally defined by extending the operational semantics underlying the program transformation language Stratego, and illustrated by means of several program transformations in Stratego, including constant propagation, bound variable renaming, dead code elimination, function inlining, and function specialization. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Eelco Visser. +
    + +
    + + + Dagstuhl 2003 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + Stratego/XT is a framework for the development of transformation systems aiming to support a wide range of program transformations. The framework consists of the transformation language Stratego and the XT collection of transformation tools. Stratego is based on the paradigm of rewriting under the control of programmable rewriting strategies. The XT tools provide facilities for the infrastructure of transformation systems including parsing and pretty-printing. The framework addresses the entire range of the development process; from the specification of transformations to their composition into transformation systems. This chapter gives an overview of the main ingredients involved in the composition of transformation systems with Stratego/XT, where we distinguish the abstraction levels of rules, strategies, tools, and systems. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Eelco Visser, Zine-El-Abidine Benaissa, Andrew P. Tolmach. +
    + +
    + + + ICFP 1998 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + We describe a language for defining term rewriting strategies, and its application to the production of program optimizers. Valid transformations on program terms can be described by a set of rewrite rules; rewriting strategies are used to describe when and how the various rules should be applied in order to obtain the desired optimization effects. Separating rules from strategies in this fashion makes it easier to reason about the behavior of the optimizer as a whole, compared to traditional monolithic optimizer implementations. We illustrate the expressiveness of our language by using it to describe a simple optimizer for an ML-like intermediate representation.The basic strategy language uses operators such as sequential composition, choice, and recursion to build transformers from a set of labeled unconditional rewrite rules. We also define an extended language in which the side-conditions and contextual rules that arise in realistic optimizer specifications can themselves be expressed as strategy-driven rewrites. We show that the features of the basic and extended languages can be expressed by breaking down the rewrite rules into their primitive building blocks, namely matching and building terms in variable binding environments. This gives us a low-level core language which has a clear semantics, can be implemented straightforwardly and can itself be optimized. The current implementation generates C code from a strategy specification. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/11.html b/lecture/11.html new file mode 100644 index 0000000..2c441b9 --- /dev/null +++ b/lecture/11.html @@ -0,0 +1,908 @@ + + + + + + + + + Lecture 11: Nano-Pass Compiler Architecture & Composing Transformations with Strategy Combinators + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 11: Nano-Pass Compiler Architecture & Composing Transformations with Strategy Combinators + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ November 18, 2021 +
+
+
+
+ +
+
+

In this lecture we take a first look at Jeremy Siek’s nano-pass compiler architecture.

+ +

Furthermore, we study how rewrite rules and strategies are composed from basic strategy combinators. The basic building blocks of transformation are matching and building terms. These are composed using combinators including sequential composition and (left) choice into more complext strategies. To define traversal strategies that apply transformations throughout a term, we define one-level traversal primitives. These can be used to define a variety of traversals, including top-down, bottom-up, and innermost.

+ +

Slides

+ + + +

References

+ + + + +
    + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Essentials of Compilation. An Incremental Approach in Racket + +
    + +
    + Jeremy G. Siek. +
    + +
    + + + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Martin Bravenboer, Arthur van Dam, Karina Olmos, Eelco Visser. +
    + +
    + + FUIN 69(1-2) 2006 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + The applicability of term rewriting to program transformation is limited by the lack of control over rule application and by the context-free nature of rewrite rules. The first problem is addressed by languages supporting user-definable rewriting strategies. The second problem is addressed by the extension of rewriting strategies with scoped dynamic rewrite rules. Dynamic rules are defined at run-time and can access variables available from their definition context. Rules defined within a rule scope are automatically retracted at the end of that scope. In this paper, we explore the design space of dynamic rules, and their application to transformation problems. The technique is formally defined by extending the operational semantics underlying the program transformation language Stratego, and illustrated by means of several program transformations in Stratego, including constant propagation, bound variable renaming, dead code elimination, function inlining, and function specialization. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Eelco Visser, Zine-El-Abidine Benaissa, Andrew P. Tolmach. +
    + +
    + + + ICFP 1998 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + We describe a language for defining term rewriting strategies, and its application to the production of program optimizers. Valid transformations on program terms can be described by a set of rewrite rules; rewriting strategies are used to describe when and how the various rules should be applied in order to obtain the desired optimization effects. Separating rules from strategies in this fashion makes it easier to reason about the behavior of the optimizer as a whole, compared to traditional monolithic optimizer implementations. We illustrate the expressiveness of our language by using it to describe a simple optimizer for an ML-like intermediate representation.The basic strategy language uses operators such as sequential composition, choice, and recursion to build transformers from a set of labeled unconditional rewrite rules. We also define an extended language in which the side-conditions and contextual rules that arise in realistic optimizer specifications can themselves be expressed as strategy-driven rewrites. We show that the features of the basic and extended languages can be expressed by breaking down the rewrite rules into their primitive building blocks, namely matching and building terms in variable binding environments. This gives us a low-level core language which has a clear semantics, can be implemented straightforwardly and can itself be optimized. The current implementation generates C code from a strategy specification. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/12.html b/lecture/12.html new file mode 100644 index 0000000..60aac15 --- /dev/null +++ b/lecture/12.html @@ -0,0 +1,742 @@ + + + + + + + + + Lecture 12: Instruction Selection & Register Allocation + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 12: Instruction Selection & Register Allocation + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ November 25, 2021 +
+
+
+
+ +
+
+

Instruction Selection

+ +

In the first part of this lecture we study target machine architecture and their instruction set, and we start looking and code generation. We study the operational semantics rules for arithmetic expressions in ChocoPy. And we explore the translation of arithmetic expressions consisting of integer constants and additions to RISC-V code. The code is somewhat naive, since it can run out of temporary registers. Possible solutions are to store temporary values on the stack or the use register allocation. +We end with looking compilation schemas as an abstraction from the implementation details of transformation rules.

+ + + +

Register Allocation

+ +

In the second part of the lecture we look at register allocation, i.e. mapping symbolic variables to concrete machine registers. +An interference graph represents the (simultaneous) use of variables, which we obtain from liveness analysis. +Using graph coloring we assign concrete registers to variables such that no two variables that are live at the same time are assigned to the same register. +When that is not possible variables need to be spilled to memory.

+ + + +

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/13.html b/lecture/13.html new file mode 100644 index 0000000..58180ea --- /dev/null +++ b/lecture/13.html @@ -0,0 +1,757 @@ + + + + + + + + + Lecture 13: Implementing Register Allocation & Control Flow & Memory Management + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 13: Implementing Register Allocation & Control Flow & Memory Management + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ December 02, 2021 +
+
+
+
+ +
+
+

Implementing Register Allocation

+ +

Last week we looked at the approach for using graph coloring for register allocation, conceptually. +This week we study Stratego code for implementing register allocation (for straigh line code). +Register allocation consists of liveness analysis, building an interference graph from the results of that analysis, coloring the nodes of that graph, and finally applying the coloring result to the program.

+ +

Control Flow

+ +

We extend our compiler to handle boolean values and control flow constructs in the subject language. +By desugaring the language we can reduce the constructs that the downstream compiler has to consider.

+ +

Memory Management

+ +

We make a start with the safety considerations of memory management and consider automated memory management techniques. +We study a range of garbage collection algorithms and study their properties. +To be continued next week.

+ +

Slides:

+ + +

+ +

References

+ +

See the memory management slide for references.

+ + + + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/14.html b/lecture/14.html new file mode 100644 index 0000000..a072e57 --- /dev/null +++ b/lecture/14.html @@ -0,0 +1,745 @@ + + + + + + + + + Lecture 14: More Memory Management & While Loops & Nested Functions + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 14: More Memory Management & While Loops & Nested Functions + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ December 09, 2021 +
+
+
+
+ +
+
+

Memory Management

+ +

In this lecture we continue the previous lecture on memory managagement in which we study memory safety, automated memory management, and several garbage collection algorithms. +See the previous lecture for the slides.

+ +

While Loops

+ +

We extend the study of control-flow statements by looking at the translation of while loops. +Loops do not need additional constructs in the intermediate language since the can be expressed using (conditional) jumps. +But register allocation for loops requires the extension of liveness analysis to compute a fixpoint of the analysis results.

+ +

Dynamic Rules in Stratego

+ +

To express context-sensitive transformations, Stratego provides dynamic rewrite rules. +We provide several examples that are useful in your compiler.

+ +

Nested Functions

+ +

We then discuss code generation for functions, including:

+ +
    +
  • The definition of a calling convention and that translates to building up a call frame.
    • +
  • Renaming of functions and variables in order to get a program with unique names in order to deal with shadowing of names.
    • +
  • The treatment of nested functions by adding a static link as extra parameter to nested functions, in order to provide access to variables in the call frame of an enclosing function. We show discuss a number of critical examples that point out issues to take into account. We illustrate a program transformation that makes explicit the number of static links to traverse in order to access the relevant call frame.
    • +
  • A challenge to exend ChocoPy with functions as first-class citizens.
    • +
+ +

Slides

+ + + + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/15.html b/lecture/15.html new file mode 100644 index 0000000..0b041a8 --- /dev/null +++ b/lecture/15.html @@ -0,0 +1,719 @@ + + + + + + + + + Lecture 15: Representing Objects + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 15: Representing Objects + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ December 16, 2021 +
+
+
+
+ +
+
+

In the first part of this lecture we study the implementation of objects.

+ +

We look at classes in ChocoPy with inheritance of attributes and methods, dynamic dispatch of method calls, object construction, and object initialization using __init__ methods.

+ +

Objects are represented in memory in two parts. +The encoding of a specific object stores its data attributes, the type of the object and its size. +Since all objects of a type share the same methods, those are not store in the object itself, but in a separate dispatch table. +The object representation includes a pointer to that dispatch table.

+ +

Implementing a class is a matter of creating a prototype object for the class, including a link to a separate dispatch table. +Allocating an object is then a matter of copying the prototype. +Each becomes a regular top-level function that takes the receiver object as an additional parameter. +Accessing the attributes of a class requires its offset with respect to the object pointer.

+ +

In the second part of the lecture we make a start with the study of data-flow analysis.

+ + + + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/16.html b/lecture/16.html new file mode 100644 index 0000000..cd5344e --- /dev/null +++ b/lecture/16.html @@ -0,0 +1,934 @@ + + + + + + + + + Lecture 16: Data-Flow Analysis + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 16: Data-Flow Analysis + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ December 23, 2021 +
+
+
+
+ +
+
+

In this lecture we study data-flow analysis and the mathmetical framework of monotone frameworks, a foundation for the definition of data-flow analyses.

+ +

We also introduce the FlowSpec language for declarative definition of intra-procedural, flow-sensitive data-flow analyses. +FlowSpec provides rules for mapping an AST to a control-flow graph and for defining the transfer functions associated with nodes in the graph.

+ +

References

+ + + + +
    + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Anders Møller, Michael I. Schwartzbach. +
    + +
    + + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + Static program analysis is the art of reasoning about the behavior of computer programs without actually running them. This is useful not only in optimizing compilers for producing efficient code but also for automatic error detection and other tools that can help programmers. As known from Turing and Rice, all interesting properties of the behavior of programs written in common programming languages are mathematically undecidable. This means that automated reasoning of software generally must involve approximation. It is also well known that testing may reveal errors but not show their absence. In contrast, static program analysis can - with the right kind of approximations - check all possible executions of the programs and provide guarantees about their properties. The challenge when developing such analyses is how to ensure high precision and efficiency to be practically useful. This teaching material concisely presents the essential principles and algorithms for static program analysis. We emphasize a constraint-based approach where suitable constraint systems conceptually divide analysis into a front-end that generates constraints from program code and a back-end that solves the constraints to produce the analysis results. The style of presentation is intended to be precise but not overly formal. The readers are assumed to be familiar with advanced programming language concepts and the basics of compiler construction. The concepts are explained using a tiny imperative programming language, TIP, which suffices to illustrate the main challenges that arise with mainstream languages. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Jeff Smits, Guido Wachsmuth, Eelco Visser. +
    + +
    + + JCL (JVLC) 57 2020 + + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + Data-flow analysis is the static analysis of programs to estimate their approximate run-time behavior or approximate intermediate run-time values. It is an integral part of modern language specifications and compilers. In the specification of static semantics of programming languages, the concept of data-flow allows the description of well-formedness such as definite assignment of a local variable before its first use. In the implementation of compiler back-ends, data-flow analyses inform optimizations. Data-flow analysis has an established theoretical foundation. What lags behind is implementations of data-flow analysis in compilers, which are usually ad-hoc. This makes such implementations difficult to extend and maintain. In previous work researchers have proposed higher-level formalisms suitable for whole-program analysis in a separate tool, incremental analysis within editors, or bound to a specific intermediate representation. In this paper, we present FlowSpec, an executable formalism for specification of data-flow analysis. FlowSpec is a domain-specific language that enables direct and concise specification of data-flow analysis for programming languages, designed to express flow-sensitive, intra-procedural analyses. We define the formal semantics of FlowSpec in terms of monotone frameworks. We describe the design of FlowSpec using examples of standard analyses. We also include a description of our implementation of FlowSpec. In a case study we evaluate FlowSpec with the static analyses for Green-Marl, a domain-specific programming language for graph analytics. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Jeff Smits, Eelco Visser. +
    + +
    + + + SLE 2017 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + We present FlowSpec, a declarative specification language for the domain of dataflow analysis. FlowSpec has declarative support for the specification of control flow graphs of programming languages, and dataflow analyses on these control flow graphs. We define the formal semantics of FlowSpec, which is rooted in Monotone Frameworks. We also discuss a prototype implementation of the language, built in the Spoofax Language Workbench. Finally, we evaluate the expressiveness and conciseness of the language with two case studies. These case studies are analyses for Green-Marl, an industrial, domain-specific language for graph processing. The first case study is a classical dataflow analysis, scaled to this full language. The second case study is a domain-specific analysis of Green-Marl. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Flemming Nielson, Hanne Riis Nielson, Chris Hankin. +
    + +
    + + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/17.html b/lecture/17.html new file mode 100644 index 0000000..cef7792 --- /dev/null +++ b/lecture/17.html @@ -0,0 +1,756 @@ + + + + + + + + + Lecture 17: CS4200-B: Code Generation Mechanics, Summary, Further Study + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 17: CS4200-B: Code Generation Mechanics, Summary, Further Study + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ January 14, 2022 +
+
+
+
+ +
+
+

Code Generation Mechanics

+ +

In the third part of the lecture, we take a broader look at mechanics for code generation, the properties we would like compilers to adhere to, and to what extend existing mechanisms support the verification of those properties out of the box.

+ +
    +
  • Code generation by string manipulation +
      +
    • Stratego does support string templates, which can be useful to do quick code generation
      • +
    +
    • +
  • Code generation by term transformation +
      +
    • This is what we do in this course. A
      • +
    +
    • +
  • Guaranteeing syntactic correct target code +
      +
    • By means of type checking terms against signature; coming up for Stratego [SLE 2020]
      • +
    +
    • +
  • Program transformation with concrete object syntax +
      +
    • Quoting concrete syntax, while transforming the underlying abstract syntax; see [GPCE 2002, OOPSLA 2004, SCP 2010]
      • +
    +
    • +
  • Hygienic transformations (avoiding name capture) +
      +
    • Hygienic macros in Scheme/Racket ensure capture avoidance in binding constructs introduced by macros
      • +
    • The namefix approach guarantees capture avoidance by checking after the fact
      • +
    • We are working on avoiding capture in rename refactoring based on scope graphs
      • +
    +
    • +
  • Guaranteeing type correct target code +
      +
    • In intrinsically-typed interpreters/compilers, the type safety of the transformation is guaranteed by construction; see [POPL 2018, 2021] for explorations of this approach in Agda
      • +
    +
    • +
  • Preservation of dynamic semantics +
      +
    • The CompCert certified compiler proves that its code generation preserves the semantics of the source language in target programs; a challenge is how to realize such certified compilation with minimal effort
      • +
    +
    • +
+ +

We only briefly touch on these topics. See the references for further information.

+ +

Slides

+ + + +

Summary

+ +

In the second part of the lecture we summarize the course and look at further study of compilers and programming languages. +In particular, we list current research challenges in the language workbench group at TU Delft, and we list further courses to take in the master program.

+ +

Slides

+ + + + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/2.html b/lecture/2.html new file mode 100644 index 0000000..1b65207 --- /dev/null +++ b/lecture/2.html @@ -0,0 +1,922 @@ + + + + + + + + + Lecture 2: Declarative Syntax Definition + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 2: Declarative Syntax Definition + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ September 03, 2021 +
+
+
+
+ +
+
+

In this lecture we study declarative syntax definition.

+ +

Slides

+ + + +

Reading Material

+ +

The [SEFM 2020] paper gives an overview of SDF3, the language we will use for syntax definition.

+ +

The [LDTA 2012] paper introduced ‘template-based syntax definition’. (This was one of the results of the master’s thesis work of Tobi Vollebregt.)

+ +

Tutorial and Talk on SDF3

+ + + +

This is part of a tutorial on Spoofax at the PLDI 2020 conference in June 2020.

+ + + +

This is a talk for the OCAML 2020 workshop at the ICFP 2020 conference.

+ +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Luis Eduardo de Souza Amorim, Eelco Visser. +
    + +
    + + + SEFM 2020 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + SDF3 is a syntax definition formalism that extends plain context-free grammars with features such as constructor declarations, declarative disambiguation rules, character-level grammars, permissive syntax, layout constraints, formatting templates, placeholder syntax, and modular composition. These features support the multi-purpose interpretation of syntax definitions, including derivation of type schemas for abstract syntax tree representations, scannerless generalized parsing of the full class of context-free grammars, error recovery, layout-sensitive parsing, parenthesization and formatting, and syntactic completion. This paper gives a high level overview of SDF3 by means of examples and provides a guide to the literature for further details. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Tobi Vollebregt, Lennart C. L. Kats, Eelco Visser. +
    + +
    + + + LDTA 2012 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Syntax discoverability has been a crucial advantage of structure editors for new users of a language. Despite this advantage, structure editors have not been widely adopted. Based on immediate parsing and analyses, modern textual code editors are also increasingly syntax-aware: structure and textual editors are converging into a new editing paradigm that combines text and templates. Current text-based language workbenches require redundant specification of the ingredients for a template-based editor, which is detrimental to the quality of syntactic completion, as consistency and completeness of the definition cannot be guaranteed. In this paper we describe the design and implementation of a specification language for syntax definition based on templates. It unifies the specification of parsers, unparsers and template-based editors. We evaluate the template language by application to two domain-specific languages used for tax benefits and mobile applications. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/3.html b/lecture/3.html new file mode 100644 index 0000000..59354b2 --- /dev/null +++ b/lecture/3.html @@ -0,0 +1,951 @@ + + + + + + + + + Lecture 3: Disambiguation and Layout Sensitive Syntax + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 3: Disambiguation and Layout Sensitive Syntax + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ September 09, 2021 +
+
+
+
+ +
+
+

In this lecture we study the declarative disambiguation of declarative syntax definitions using associativity and priority rules. +As a prelimary to understanding disambiguation, we first study the semantics of context-free grammars in terms of sentence and tree derivation. +We discuss the semantics of such rules in terms of subtree exclusion and how (simple) disambiguations can be expressed by means of grammar transformation.

+ +

We end with a discussion of disambiguation using layout constraints, in order to express layout-sensitive syntax.

+ +

Slides

+ + + +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Luis Eduardo de Souza Amorim, Michael J. Steindorfer, Sebastian Erdweg, Eelco Visser. +
    + +
    + + + SLE 2018 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + In layout-sensitive languages, the indentation of an expression or statement can influence how a program is parsed. While some of these languages (e.g., Haskell and Python) have been widely adopted, there is little support for software language engineers in building tools for layout-sensitive languages. As a result, parsers, pretty-printers, program analyses, and refactoring tools often need to be handwritten, which decreases the maintainability and extensibility of these tools. Even state-of-the-art language workbenches have little support for layout-sensitive languages, restricting the development and prototyping of such languages. In this paper, we introduce a novel approach to declarative specification of layout-sensitive languages using layout declarations. Layout declarations are high-level specifications of indentation rules that abstract from low-level technicalities. We show how to derive an efficient layout-sensitive generalized parser and a corresponding pretty-printer automatically from a language specification with layout declarations. We validate our approach in a case-study using a syntax definition for the Haskell programming language, investigating the performance of the generated parser and the correctness of the generated pretty-printer against 22191 Haskell files. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Sebastian Erdweg, Tillmann Rendel, Christian Kästner, Klaus Ostermann. +
    + +
    + + + SLE 2012 + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Sebastian Erdweg, Felix Rieger, Tillmann Rendel, Klaus Ostermann. +
    + +
    + + + haskell 2012 + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/4.html b/lecture/4.html new file mode 100644 index 0000000..bcc978f --- /dev/null +++ b/lecture/4.html @@ -0,0 +1,967 @@ + + + + + + + + + Lecture 4: Type Checking and Type Constraints + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 4: Type Checking and Type Constraints + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ September 16, 2021 +
+
+
+
+ +
+
+

In this lecture we study types and relations in types in programming languages, and discuss the motivation for having types.

+ +

Next we look at the way type systems are formalized. In particular, we look at the inference rules in the ChocoPy reference manual.

+ +

Finally, we start our study of the Statix meta-language for the specification of type systems. Ending with a demonstration of the Chicago project (available on github), as an example of the use of Statix.

+ +

There are no papers that give concrete examples of Statix code, but the PEPM 2016 paper gives a good introduction to the use of constraints and scope graphs, albeit in a somewhat more limited language than Statix.

+ +

Slides

+ + + +

Statix Tutorial

+ +

This is a recording of a tutorial on Statix at the PLDI conference.

+ + + +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Scopes as types + +
    + +
    + Hendrik van Antwerpen, Casper Bach Poulsen, Arjen Rouvoet, Eelco Visser. +
    + +
    + + PACMPL 2(OOPSLA) 2018 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Scope graphs are a promising generic framework to model the binding structures of programming languages, bridging formalization and implementation, supporting the definition of type checkers and the automation of type safety proofs. However, previous work on scope graphs has been limited to simple, nominal type systems. In this paper, we show that viewing scopes as types enables us to model the internal structure of types in a range of non-simple type systems (including structural records and generic classes) using the generic representation of scopes. Further, we show that relations between such types can be expressed in terms of generalized scope graph queries. We extend scope graphs with scoped relations and queries. We introduce Statix, a new domain-specific meta-language for the specification of static semantics, based on scope graphs and constraints. We evaluate the scopes as types approach and the Statix design in case studies of the simply-typed lambda calculus with records, System F, and Featherweight Generic Java. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Hendrik van Antwerpen, Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + PEPM 2016 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + In previous work, we introduced scope graphs as a formalism for describing program binding structure and performing name resolution in an AST-independent way. In this paper, we show how to use scope graphs to build static semantic analyzers. We use constraints extracted from the AST to specify facts about binding, typing, and initialization. We treat name and type resolution as separate building blocks, but our approach can handle language constructs -- such as record field access -- for which binding and typing are mutually dependent. We also refine and extend our previous scope graph theory to address practical concerns including ambiguity checking and support for a wider range of scope relationships. We describe the details of constraint generation for a model language that illustrates many of the interesting static analysis issues associated with modules and records. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + ESOP 2015 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + We describe a language-independent theory for name binding and resolution, suitable for programming languages with complex scoping rules including both lexical scoping and modules. We formulate name resolution as a two-stage problem. First a language-independent scope graph is constructed using language-specific rules from an abstract syntax tree. Then references in the scope graph are resolved to corresponding declarations using a language-independent resolution process. We introduce a resolution calculus as a concise, declarative, and languageindependent specification of name resolution. We develop a resolution algorithm that is sound and complete with respect to the calculus. Based on the resolution calculus we develop language-independent definitions of α-equivalence and rename refactoring. We illustrate the approach using a small example language with modules. In addition, we show how our approach provides a model for a range of name binding patterns in existing languages. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/5.html b/lecture/5.html new file mode 100644 index 0000000..9540ffe --- /dev/null +++ b/lecture/5.html @@ -0,0 +1,974 @@ + + + + + + + + + Lecture 5: Name Binding and Name Resolution + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 5: Name Binding and Name Resolution + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ September 23, 2021 +
+
+
+
+ +
+
+

In this lecture we study name binding and name resolution. We introduce the notion of a scope graph to represent the bindings in a program and name resolution queries in a scope graph to resolve references (uses of names) to declarations (definitions of names).

+ +

In addition to the lecture (see the recording on BrightSpace), see the talks about scope graphs below. +The Curry On 2017 talk provides a somewhat higher-level presentation of the resolution calculus that is at the basis of resolution scope graphs. +See also to Statix tutorial listed on the page of the previous lecture.

+ +

Slides

+ + + +

Scopes as Types

+ +

Presentation of the OOPSLA’18 paper that introduced Statix.

+ + + +

Scope Graphs

+ +

A high-level introduction to scope graphs and the resolution calculus.

+ + + +

Towards Semantic Editor Services

+ +

Explores alternative interpretation of Statix specifications for editor services such as semantic completion.

+ + + +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Scopes as types + +
    + +
    + Hendrik van Antwerpen, Casper Bach Poulsen, Arjen Rouvoet, Eelco Visser. +
    + +
    + + PACMPL 2(OOPSLA) 2018 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Scope graphs are a promising generic framework to model the binding structures of programming languages, bridging formalization and implementation, supporting the definition of type checkers and the automation of type safety proofs. However, previous work on scope graphs has been limited to simple, nominal type systems. In this paper, we show that viewing scopes as types enables us to model the internal structure of types in a range of non-simple type systems (including structural records and generic classes) using the generic representation of scopes. Further, we show that relations between such types can be expressed in terms of generalized scope graph queries. We extend scope graphs with scoped relations and queries. We introduce Statix, a new domain-specific meta-language for the specification of static semantics, based on scope graphs and constraints. We evaluate the scopes as types approach and the Statix design in case studies of the simply-typed lambda calculus with records, System F, and Featherweight Generic Java. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Hendrik van Antwerpen, Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + PEPM 2016 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + In previous work, we introduced scope graphs as a formalism for describing program binding structure and performing name resolution in an AST-independent way. In this paper, we show how to use scope graphs to build static semantic analyzers. We use constraints extracted from the AST to specify facts about binding, typing, and initialization. We treat name and type resolution as separate building blocks, but our approach can handle language constructs -- such as record field access -- for which binding and typing are mutually dependent. We also refine and extend our previous scope graph theory to address practical concerns including ambiguity checking and support for a wider range of scope relationships. We describe the details of constraint generation for a model language that illustrates many of the interesting static analysis issues associated with modules and records. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + ESOP 2015 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + We describe a language-independent theory for name binding and resolution, suitable for programming languages with complex scoping rules including both lexical scoping and modules. We formulate name resolution as a two-stage problem. First a language-independent scope graph is constructed using language-specific rules from an abstract syntax tree. Then references in the scope graph are resolved to corresponding declarations using a language-independent resolution process. We introduce a resolution calculus as a concise, declarative, and languageindependent specification of name resolution. We develop a resolution algorithm that is sound and complete with respect to the calculus. Based on the resolution calculus we develop language-independent definitions of α-equivalence and rename refactoring. We illustrate the approach using a small example language with modules. In addition, we show how our approach provides a model for a range of name binding patterns in existing languages. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/6.html b/lecture/6.html new file mode 100644 index 0000000..40a4366 --- /dev/null +++ b/lecture/6.html @@ -0,0 +1,1063 @@ + + + + + + + + + Lecture 6: Constraint Semantics and Constraint Resolution + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 6: Constraint Semantics and Constraint Resolution + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ September 30, 2021 +
+
+
+
+ +
+
+

In this lecture we study the semantics and implementation of type checking with constraints. +We study the declarative semantics of constraints by means of constraint satisfaction: given a model (solution), does is the constraint satisfied under that model? +And we study the implementation of constraint solvers as a rewrite system that simplifies constraints to basic unification, scope graph, and name resolution constraints. +Finally, we study unification, its properties, a naive algorithm, and efficient implementation using union-find.

+ +

The PDF of the slides with builds are useful for the union-find section.

+ +

Slides

+ + + +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Arjen Rouvoet, Hendrik van Antwerpen, Casper Bach Poulsen, Robbert Krebbers, Eelco Visser. +
    + +
    + + PACMPL 4(OOPSLA) 2020 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + There is a large gap between the specification of type systems and the implementation of their type checkers, which impedes reasoning about the soundness of the type checker with respect to the specification. A vision to close this gap is to automatically obtain type checkers from declarative programming language specifications. This moves the burden of proving correctness from a case-by-case basis for concrete languages to a single correctness proof for the specification language. This vision is obstructed by an aspect common to all programming languages: name resolution. Naming and scoping are pervasive and complex aspects of the static semantics of programming languages. Implementations of type checkers for languages with name binding features such as modules, imports, classes, and inheritance interleave collection of binding information (i.e., declarations, scoping structure, and imports) and querying that information. This requires scheduling those two aspects in such a way that query answers are stable—i.e., they are computed only after all relevant binding structure has been collected. Type checkers for concrete languages accomplish stability using language-specific knowledge about the type system. In this paper we give a language-independent characterization of necessary and sufficient conditions to guarantee stability of name and type queries during type checking in terms of critical edges in an incomplete scope graph. We use critical edges to give a formal small-step operational semantics to a declarative specification language for type systems, that achieves soundness by delaying queries that may depend on missing information. This yields type checkers for the specified languages that are sound by construction—i.e., they schedule queries so that the answers are stable, and only accept programs that are name- and type-correct according to the declarative language specification. We implement this approach, and evaluate it against specifications of a small module and record language, as well as subsets of Java and Scala. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Scopes as types + +
    + +
    + Hendrik van Antwerpen, Casper Bach Poulsen, Arjen Rouvoet, Eelco Visser. +
    + +
    + + PACMPL 2(OOPSLA) 2018 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Scope graphs are a promising generic framework to model the binding structures of programming languages, bridging formalization and implementation, supporting the definition of type checkers and the automation of type safety proofs. However, previous work on scope graphs has been limited to simple, nominal type systems. In this paper, we show that viewing scopes as types enables us to model the internal structure of types in a range of non-simple type systems (including structural records and generic classes) using the generic representation of scopes. Further, we show that relations between such types can be expressed in terms of generalized scope graph queries. We extend scope graphs with scoped relations and queries. We introduce Statix, a new domain-specific meta-language for the specification of static semantics, based on scope graphs and constraints. We evaluate the scopes as types approach and the Statix design in case studies of the simply-typed lambda calculus with records, System F, and Featherweight Generic Java. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Hendrik van Antwerpen, Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + PEPM 2016 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + In previous work, we introduced scope graphs as a formalism for describing program binding structure and performing name resolution in an AST-independent way. In this paper, we show how to use scope graphs to build static semantic analyzers. We use constraints extracted from the AST to specify facts about binding, typing, and initialization. We treat name and type resolution as separate building blocks, but our approach can handle language constructs -- such as record field access -- for which binding and typing are mutually dependent. We also refine and extend our previous scope graph theory to address practical concerns including ambiguity checking and support for a wider range of scope relationships. We describe the details of constraint generation for a model language that illustrates many of the interesting static analysis issues associated with modules and records. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + ESOP 2015 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + We describe a language-independent theory for name binding and resolution, suitable for programming languages with complex scoping rules including both lexical scoping and modules. We formulate name resolution as a two-stage problem. First a language-independent scope graph is constructed using language-specific rules from an abstract syntax tree. Then references in the scope graph are resolved to corresponding declarations using a language-independent resolution process. We introduce a resolution calculus as a concise, declarative, and languageindependent specification of name resolution. We develop a resolution algorithm that is sound and complete with respect to the calculus. Based on the resolution calculus we develop language-independent definitions of α-equivalence and rename refactoring. We illustrate the approach using a small example language with modules. In addition, we show how our approach provides a model for a range of name binding patterns in existing languages. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Unification Theory + +
    + +
    + Franz Baader, Wayne Snyder. +
    + +
    + + + + Handbook of Automated Reasoning (in 2 volumes) 2001 + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/7.html b/lecture/7.html new file mode 100644 index 0000000..91e8ed7 --- /dev/null +++ b/lecture/7.html @@ -0,0 +1,1039 @@ + + + + + + + + + Lecture 7: More Constraints and Statix + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Lecture 7: More Constraints and Statix + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + +
+
+ October 07, 2021 +
+
+
+
+ +
+
+

In this lecture we discuss material we didn’t get to in the previous two lectures.

+ +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Arjen Rouvoet, Hendrik van Antwerpen, Casper Bach Poulsen, Robbert Krebbers, Eelco Visser. +
    + +
    + + PACMPL 4(OOPSLA) 2020 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + There is a large gap between the specification of type systems and the implementation of their type checkers, which impedes reasoning about the soundness of the type checker with respect to the specification. A vision to close this gap is to automatically obtain type checkers from declarative programming language specifications. This moves the burden of proving correctness from a case-by-case basis for concrete languages to a single correctness proof for the specification language. This vision is obstructed by an aspect common to all programming languages: name resolution. Naming and scoping are pervasive and complex aspects of the static semantics of programming languages. Implementations of type checkers for languages with name binding features such as modules, imports, classes, and inheritance interleave collection of binding information (i.e., declarations, scoping structure, and imports) and querying that information. This requires scheduling those two aspects in such a way that query answers are stable—i.e., they are computed only after all relevant binding structure has been collected. Type checkers for concrete languages accomplish stability using language-specific knowledge about the type system. In this paper we give a language-independent characterization of necessary and sufficient conditions to guarantee stability of name and type queries during type checking in terms of critical edges in an incomplete scope graph. We use critical edges to give a formal small-step operational semantics to a declarative specification language for type systems, that achieves soundness by delaying queries that may depend on missing information. This yields type checkers for the specified languages that are sound by construction—i.e., they schedule queries so that the answers are stable, and only accept programs that are name- and type-correct according to the declarative language specification. We implement this approach, and evaluate it against specifications of a small module and record language, as well as subsets of Java and Scala. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Scopes as types + +
    + +
    + Hendrik van Antwerpen, Casper Bach Poulsen, Arjen Rouvoet, Eelco Visser. +
    + +
    + + PACMPL 2(OOPSLA) 2018 + + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + Scope graphs are a promising generic framework to model the binding structures of programming languages, bridging formalization and implementation, supporting the definition of type checkers and the automation of type safety proofs. However, previous work on scope graphs has been limited to simple, nominal type systems. In this paper, we show that viewing scopes as types enables us to model the internal structure of types in a range of non-simple type systems (including structural records and generic classes) using the generic representation of scopes. Further, we show that relations between such types can be expressed in terms of generalized scope graph queries. We extend scope graphs with scoped relations and queries. We introduce Statix, a new domain-specific meta-language for the specification of static semantics, based on scope graphs and constraints. We evaluate the scopes as types approach and the Statix design in case studies of the simply-typed lambda calculus with records, System F, and Featherweight Generic Java. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Hendrik van Antwerpen, Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + PEPM 2016 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + In previous work, we introduced scope graphs as a formalism for describing program binding structure and performing name resolution in an AST-independent way. In this paper, we show how to use scope graphs to build static semantic analyzers. We use constraints extracted from the AST to specify facts about binding, typing, and initialization. We treat name and type resolution as separate building blocks, but our approach can handle language constructs -- such as record field access -- for which binding and typing are mutually dependent. We also refine and extend our previous scope graph theory to address practical concerns including ambiguity checking and support for a wider range of scope relationships. We describe the details of constraint generation for a model language that illustrates many of the interesting static analysis issues associated with modules and records. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Pierre Néron, Andrew P. Tolmach, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + ESOP 2015 + + + + + + [pdf, doi, bib, researchr, ] + + + +
    + +
    +
    + We describe a language-independent theory for name binding and resolution, suitable for programming languages with complex scoping rules including both lexical scoping and modules. We formulate name resolution as a two-stage problem. First a language-independent scope graph is constructed using language-specific rules from an abstract syntax tree. Then references in the scope graph are resolved to corresponding declarations using a language-independent resolution process. We introduce a resolution calculus as a concise, declarative, and languageindependent specification of name resolution. We develop a resolution algorithm that is sound and complete with respect to the calculus. Based on the resolution calculus we develop language-independent definitions of α-equivalence and rename refactoring. We illustrate the approach using a small example language with modules. In addition, we show how our approach provides a model for a range of name binding patterns in existing languages. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Unification Theory + +
    + +
    + Franz Baader, Wayne Snyder. +
    + +
    + + + + Handbook of Automated Reasoning (in 2 volumes) 2001 + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/8.html b/lecture/8.html new file mode 100644 index 0000000..1804999 --- /dev/null +++ b/lecture/8.html @@ -0,0 +1,989 @@ + + + + + + + + + Lecture 8a: From Context-Free Grammars to Parsers + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 8a: From Context-Free Grammars to Parsers + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ October 14, 2021 +
+
+
+
+ +
+
+

In this lecture we study how we can derive parsers from context-free grammars. +We start with looking again at the semantics of context-free grammars as derivations. +Then we observe that parsing is the inverse of derivation, i.e. reducing a sentence to a (start) symbol or parse tree. +A shift-reduce parsers computes a right-most derivation for a sentence. +The question is then how to make that process deterministic and efficient. +First we see how item sets encode the state of a shift-reduce parser and make it clear what actions can be undertaking without consulting the grammar. +Next, we compute the FOLLOW set of a grammar in order to determine which action to take next in case there are multiple possible actions in a state. +This approach yields a linear deterministic parser for grammars in the the SLR(1) format, i.e. those grammars for which this method produces a parse table without shift/reduce conflicts.

+ + + +

Slides

+ + + +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth. +
    + +
    + + + OOPSLA 2010 + + + + + + [doi, bib, researchr, ] + + + +
    + +
    +
    + Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Compilers: Principles, Techniques, and Tools (2nd Edition) + +
    + +
    + A. V. Aho, M. S. Lam, R. Sethi, J. D. Ullman. +
    + +
    + + + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + A Case Study in Optimizing Parsing Schemata by Disambiguation Filters + +
    + +
    + Eelco Visser. +
    + +
    + + + International Workshop on Parsing Technology (IWPT 1997) 1997 + + + + + + [bib, researchr, ] + + + +
    + +
    +
    + Disambiguation methods for context-free grammars enable concise specification of programming languages by ambiguous grammars. A disambiguation filter is a function that selects a subset from a set of parse trees---the possible parse trees for an ambiguous sentence. The framework of filters provides a declarative description of disambiguation methods independent of parsing. Although filters can be implemented straightforwardly as functions that prune the parse forest produced by some generalized, this can be too inefficient for practical applications. In this paper the optimization of parsing schemata, a framework for high-level description of parsing algorithms, by disambiguation filters is considered in order to find efficient parsing algorithms for declaratively specified disambiguation methods. As a case study the optimization of the parsing schema of Earley's parsing algorithm by two filters is investigated. The main result is a technique for generation of efficient LR-like parsers for ambiguous grammars modulo priorities. +
    +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Parsing Schemata + +
    + +
    + Klaas Sikkel. +
    + +
    + + + + + + + PhD thesis, University of Twente, Enschede, Netherlands, 1993 + + [pdf, doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/8b.html b/lecture/8b.html new file mode 100644 index 0000000..42efe7b --- /dev/null +++ b/lecture/8b.html @@ -0,0 +1,1100 @@ + + + + + + + + + Lecture 8b: More Parsing Algorithms + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 8b: More Parsing Algorithms + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ October 14, 2021 +
+
+
+
+ +
+
+

Note: This lecture is for the interested student and will not be part of the exam material.

+ +

In this lecture we study two more parsing algorithms.

+ +

Predictive Parsing

+ +

Predictive parsing is a top-down parsing approach in which the stack contains the symbols to parse. +Actions consist in either popping a terminal off the stack when that coincides with the next input token, or in predicting a production for the non-terminal that is on top of the stack and replacing that non-terminal with the symbols on the right-hand side of the predicted production. +We discuss how to derive an LL(1) automaton from a grammar based on the FIRST and FOLLOW set for that grammar.

+ +

Generalized Parsing

+ +

Generalized LR parsing is a generalization of the LR parsing algorithm that we studied last week. +The LR parsing algorithm is deterministic. +At each point there is a single shift or reduce action that can take place. +That means, that an LR parser cannot handle shift/reduce conflicts in the parse table. +A Generalized-LR parser handles such conflicts by forking the stack and continuing to parse the input with two parsers in parallel. +This may lead to multiple parsers to be active simultaneously. +When two parsers end up in the same state again, their stacks are merged.

+ +

Other Parsing Algorithms

+ +

There are many more parsing algorithms than we can study in this course. +In the references a couple of pointers to other sources about parsing algorithms. +These include ALL(*), the algorithm that is at the basis of the popular ANTLR parser generator, +parsing expression grammars (PEGs) another popular parsing approach, +and parsing combinators in Haskell and Scala.

+ + + +

Slides

+ + + +

References

+ + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Terence John Parr, Sam Harwell, Kathleen Fisher. +
    + +
    + + + OOPSLA 2014 + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Parser combinators in Scala + +
    + +
    + Adriaan Moors, F. Piessens, Martin Odersky. +
    + +
    + + + + + + Technical report , Department of Computer Science, K.U. Leuven, 2008 + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Compilers: Principles, Techniques, and Tools (2nd Edition) + +
    + +
    + A. V. Aho, M. S. Lam, R. Sethi, J. D. Ullman. +
    + +
    + + + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + + + +
    + Bryan Ford. +
    + +
    + + + POPL 2004 + + + + + + [doi, bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Modern Compiler Implementation in Java, 2nd edition + +
    + +
    + Andrew W. Appel. +
    + +
    + + + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + +
    + + + + + + + + + +
    + + + Higher-Order Functions for Parsing + +
    + +
    + Graham Hutton. +
    + +
    + + JFP 2(3) 1992 + + + + + + + [bib, researchr] + + + +
    + + + + + + +
    + +
    • + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lecture/9.html b/lecture/9.html new file mode 100644 index 0000000..b4bd629 --- /dev/null +++ b/lecture/9.html @@ -0,0 +1,743 @@ + + + + + + + + + Lecture 9: CS4200-A: Summary and Q&A + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + +
+
+

+ + Lecture 9: CS4200-A: Summary and Q&A + +

+
+
+ + Eelco Visser + + +
+
+ + + Lecture + + + + + + + | PDF + + +
+
+ October 21, 2021 +
+
+
+
+ +
+
+

In this lecture we look back at what we have studied in the course and I answer questions about the material.

+ +

If you have questions please submit them by Wednesday morning October 21.

+ +

Slides

+ + + + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/lectures/2021/01-introduction/CS4200-2021-1-introduction-builds.pdf b/lectures/2021/01-introduction/CS4200-2021-1-introduction-builds.pdf new file mode 100644 index 0000000..cb9a0aa Binary files /dev/null and b/lectures/2021/01-introduction/CS4200-2021-1-introduction-builds.pdf differ diff --git a/lectures/2021/01-introduction/CS4200-2021-1-introduction.pdf b/lectures/2021/01-introduction/CS4200-2021-1-introduction.pdf new file mode 100644 index 0000000..2935055 Binary files /dev/null and b/lectures/2021/01-introduction/CS4200-2021-1-introduction.pdf differ diff --git a/lectures/2021/01-introduction/CS4200-2021-1-introduction/CS4200-2021-1-introduction.001.png b/lectures/2021/01-introduction/CS4200-2021-1-introduction/CS4200-2021-1-introduction.001.png new file mode 100644 index 0000000..bbb04c1 Binary files /dev/null and b/lectures/2021/01-introduction/CS4200-2021-1-introduction/CS4200-2021-1-introduction.001.png differ diff --git a/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition-builds.pdf b/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition-builds.pdf new file mode 100644 index 0000000..bb0fa84 Binary files /dev/null and b/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition-builds.pdf differ diff --git a/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition.pdf b/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition.pdf new file mode 100644 index 0000000..78c3c8b Binary files /dev/null and b/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition.pdf differ diff --git a/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition/CS4200-2021-2-syntax-definition.001.png b/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition/CS4200-2021-2-syntax-definition.001.png new file mode 100644 index 0000000..b0a7aeb Binary files /dev/null and b/lectures/2021/02-syntax-definition/CS4200-2021-2-syntax-definition/CS4200-2021-2-syntax-definition.001.png differ diff --git a/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation-builds.pdf b/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation-builds.pdf new file mode 100644 index 0000000..bad99ff Binary files /dev/null and b/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation-builds.pdf differ diff --git a/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation.pdf b/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation.pdf new file mode 100644 index 0000000..9c07435 Binary files /dev/null and b/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation.pdf differ diff --git a/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation/CS4200-2021-3-disambiguation.001.png b/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation/CS4200-2021-3-disambiguation.001.png new file mode 100644 index 0000000..95de5ab Binary files /dev/null and b/lectures/2021/03-disambiguation/CS4200-2021-3-disambiguation/CS4200-2021-3-disambiguation.001.png differ diff --git a/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking-builds.pdf b/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking-builds.pdf new file mode 100644 index 0000000..9bd6721 Binary files /dev/null and b/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking-builds.pdf differ diff --git a/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking.pdf b/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking.pdf new file mode 100644 index 0000000..8254e3b Binary files /dev/null and b/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking.pdf differ diff --git a/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking/CS4200-2021-4-type-checking.047.png b/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking/CS4200-2021-4-type-checking.047.png new file mode 100644 index 0000000..9be178c Binary files /dev/null and b/lectures/2021/04-type-constraints/CS4200-2021-4-type-checking/CS4200-2021-4-type-checking.047.png differ diff --git a/lectures/2021/05-name-binding/CS4200-2020-5-name-binding.pdf b/lectures/2021/05-name-binding/CS4200-2020-5-name-binding.pdf new file mode 100644 index 0000000..cb5bb6e Binary files /dev/null and b/lectures/2021/05-name-binding/CS4200-2020-5-name-binding.pdf differ diff --git a/lectures/2021/05-name-binding/CS4200-2021-5-name-binding/CS4200-2021-5-name-binding.001.png b/lectures/2021/05-name-binding/CS4200-2021-5-name-binding/CS4200-2021-5-name-binding.001.png new file mode 100644 index 0000000..d5c2b7d Binary files /dev/null and b/lectures/2021/05-name-binding/CS4200-2021-5-name-binding/CS4200-2021-5-name-binding.001.png differ diff --git a/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints-builds.pdf b/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints-builds.pdf new file mode 100644 index 0000000..bda5872 Binary files /dev/null and b/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints-builds.pdf differ diff --git a/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints.pdf b/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints.pdf new file mode 100644 index 0000000..bf64805 Binary files /dev/null and b/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints.pdf differ diff --git a/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints/CS4200-2021-6-solving-constraints.036.png b/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints/CS4200-2021-6-solving-constraints.036.png new file mode 100644 index 0000000..07b8d5a Binary files /dev/null and b/lectures/2021/06-solving-constraints/CS4200-2021-6-solving-constraints/CS4200-2021-6-solving-constraints.036.png differ diff --git a/lectures/2021/07-parsing/CS4200-2020-7-parsing-slr-table.png b/lectures/2021/07-parsing/CS4200-2020-7-parsing-slr-table.png new file mode 100644 index 0000000..33902c2 Binary files /dev/null and b/lectures/2021/07-parsing/CS4200-2020-7-parsing-slr-table.png differ diff --git a/lectures/2021/07-parsing/CS4200-2021-7-parsing-builds.pdf b/lectures/2021/07-parsing/CS4200-2021-7-parsing-builds.pdf new file mode 100644 index 0000000..cf881cf Binary files /dev/null and b/lectures/2021/07-parsing/CS4200-2021-7-parsing-builds.pdf differ diff --git a/lectures/2021/07-parsing/CS4200-2021-7-parsing.pdf b/lectures/2021/07-parsing/CS4200-2021-7-parsing.pdf new file mode 100644 index 0000000..57a6ca3 Binary files /dev/null and b/lectures/2021/07-parsing/CS4200-2021-7-parsing.pdf differ diff --git a/lectures/2021/08-more-parsing/CS4200-2020-8-parsing-builds.pdf b/lectures/2021/08-more-parsing/CS4200-2020-8-parsing-builds.pdf new file mode 100644 index 0000000..1dda284 Binary files /dev/null and b/lectures/2021/08-more-parsing/CS4200-2020-8-parsing-builds.pdf differ diff --git a/lectures/2021/08-more-parsing/CS4200-2020-8-parsing.pdf b/lectures/2021/08-more-parsing/CS4200-2020-8-parsing.pdf new file mode 100644 index 0000000..bc399e0 Binary files /dev/null and b/lectures/2021/08-more-parsing/CS4200-2020-8-parsing.pdf differ diff --git a/lectures/2021/08-more-parsing/title.png b/lectures/2021/08-more-parsing/title.png new file mode 100644 index 0000000..a53978e Binary files /dev/null and b/lectures/2021/08-more-parsing/title.png differ diff --git a/lectures/2021/09-summary/CS4200-2021-9-summary-builds.pdf b/lectures/2021/09-summary/CS4200-2021-9-summary-builds.pdf new file mode 100644 index 0000000..735a19c Binary files /dev/null and b/lectures/2021/09-summary/CS4200-2021-9-summary-builds.pdf differ diff --git a/lectures/2021/09-summary/CS4200-2021-9-summary.pdf b/lectures/2021/09-summary/CS4200-2021-9-summary.pdf new file mode 100644 index 0000000..194d33f Binary files /dev/null and b/lectures/2021/09-summary/CS4200-2021-9-summary.pdf differ diff --git a/lectures/2021/09-summary/CS4200-2021-9-summary/CS4200-2021-9-summary.001.png b/lectures/2021/09-summary/CS4200-2021-9-summary/CS4200-2021-9-summary.001.png new file mode 100644 index 0000000..fa6e43b Binary files /dev/null and b/lectures/2021/09-summary/CS4200-2021-9-summary/CS4200-2021-9-summary.001.png differ diff --git a/lectures/2021/10-transformation/CS4200-2021-10-transformation.pdf b/lectures/2021/10-transformation/CS4200-2021-10-transformation.pdf new file mode 100644 index 0000000..57c3aaa Binary files /dev/null and b/lectures/2021/10-transformation/CS4200-2021-10-transformation.pdf differ diff --git a/lectures/2021/10-transformation/CS4200-2021-10-transformation/CS4200-2021-10-transformation.001.png b/lectures/2021/10-transformation/CS4200-2021-10-transformation/CS4200-2021-10-transformation.001.png new file mode 100644 index 0000000..375d075 Binary files /dev/null and b/lectures/2021/10-transformation/CS4200-2021-10-transformation/CS4200-2021-10-transformation.001.png differ diff --git a/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators-builds.pdf b/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators-builds.pdf new file mode 100644 index 0000000..78e21dc Binary files /dev/null and b/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators-builds.pdf differ diff --git a/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators.pdf b/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators.pdf new file mode 100644 index 0000000..4865948 Binary files /dev/null and b/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators.pdf differ diff --git a/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators/CS4200-2021-11-strategy-combinators.001.png b/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators/CS4200-2021-11-strategy-combinators.001.png new file mode 100644 index 0000000..2710482 Binary files /dev/null and b/lectures/2021/11-transformation/CS4200-2021-11-strategy-combinators/CS4200-2021-11-strategy-combinators.001.png differ diff --git a/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation-builds.pdf b/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation-builds.pdf new file mode 100644 index 0000000..e9e91df Binary files /dev/null and b/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation-builds.pdf differ diff --git a/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation.pdf b/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation.pdf new file mode 100644 index 0000000..481b051 Binary files /dev/null and b/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation.pdf differ diff --git a/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation/CS4200-2021-12-code-generation.001.png b/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation/CS4200-2021-12-code-generation.001.png new file mode 100644 index 0000000..1d8d167 Binary files /dev/null and b/lectures/2021/12-instruction-selection/CS4200-2021-12-code-generation/CS4200-2021-12-code-generation.001.png differ diff --git a/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation-builds.pdf b/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation-builds.pdf new file mode 100644 index 0000000..e79aadc Binary files /dev/null and b/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation-builds.pdf differ diff --git a/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation.pdf b/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation.pdf new file mode 100644 index 0000000..e380a4e Binary files /dev/null and b/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation.pdf differ diff --git a/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation/CS4200-2021-12-register-allocation.030.png b/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation/CS4200-2021-12-register-allocation.030.png new file mode 100644 index 0000000..64ec656 Binary files /dev/null and b/lectures/2021/12-instruction-selection/CS4200-2021-12-register-allocation/CS4200-2021-12-register-allocation.030.png differ diff --git a/lectures/2021/13-control-flow/CS4200-2021-13-control-flow.pdf b/lectures/2021/13-control-flow/CS4200-2021-13-control-flow.pdf new file mode 100644 index 0000000..bc11215 Binary files /dev/null and b/lectures/2021/13-control-flow/CS4200-2021-13-control-flow.pdf differ diff --git a/lectures/2021/13-control-flow/CS4200-2021-13-control-flow/CS4200-2021-13-control-flow.011.png b/lectures/2021/13-control-flow/CS4200-2021-13-control-flow/CS4200-2021-13-control-flow.011.png new file mode 100644 index 0000000..2bf3188 Binary files /dev/null and b/lectures/2021/13-control-flow/CS4200-2021-13-control-flow/CS4200-2021-13-control-flow.011.png differ diff --git a/lectures/2021/13-control-flow/CS4200-2021-13-memory-management.pdf b/lectures/2021/13-control-flow/CS4200-2021-13-memory-management.pdf new file mode 100644 index 0000000..9b7b1a3 Binary files /dev/null and b/lectures/2021/13-control-flow/CS4200-2021-13-memory-management.pdf differ diff --git a/lectures/2021/13-control-flow/CS4200-2021-13-memory-management/CS4200-2021-13-memory-management.022.png b/lectures/2021/13-control-flow/CS4200-2021-13-memory-management/CS4200-2021-13-memory-management.022.png new file mode 100644 index 0000000..f247297 Binary files /dev/null and b/lectures/2021/13-control-flow/CS4200-2021-13-memory-management/CS4200-2021-13-memory-management.022.png differ diff --git a/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions-builds.pdf b/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions-builds.pdf new file mode 100644 index 0000000..c6d3486 Binary files /dev/null and b/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions-builds.pdf differ diff --git a/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions.pdf b/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions.pdf new file mode 100644 index 0000000..d67ae35 Binary files /dev/null and b/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions.pdf differ diff --git a/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions/CS4200-2021-14-nested-functions.006.png b/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions/CS4200-2021-14-nested-functions.006.png new file mode 100644 index 0000000..25005e0 Binary files /dev/null and b/lectures/2021/14-nested-functions/CS4200-2021-14-nested-functions/CS4200-2021-14-nested-functions.006.png differ diff --git a/lectures/2021/15-objects/CS4200-2021-15-objects.pdf b/lectures/2021/15-objects/CS4200-2021-15-objects.pdf new file mode 100644 index 0000000..21f69bb Binary files /dev/null and b/lectures/2021/15-objects/CS4200-2021-15-objects.pdf differ diff --git a/lectures/2021/15-objects/CS4200-2021-15-objects/CS4200-2021-15-objects.005.png b/lectures/2021/15-objects/CS4200-2021-15-objects/CS4200-2021-15-objects.005.png new file mode 100644 index 0000000..6008d6a Binary files /dev/null and b/lectures/2021/15-objects/CS4200-2021-15-objects/CS4200-2021-15-objects.005.png differ diff --git a/lectures/2021/16-data-flow-analysis/CS4200-2021-16-data-flow-analysis.pdf b/lectures/2021/16-data-flow-analysis/CS4200-2021-16-data-flow-analysis.pdf new file mode 100644 index 0000000..6603fc9 Binary files /dev/null and b/lectures/2021/16-data-flow-analysis/CS4200-2021-16-data-flow-analysis.pdf differ diff --git a/lectures/2021/16-data-flow-analysis/CS4200-2021-16-data-flow-analysis/CS4200-2021-16-data-flow-analysis.031.png b/lectures/2021/16-data-flow-analysis/CS4200-2021-16-data-flow-analysis/CS4200-2021-16-data-flow-analysis.031.png new file mode 100644 index 0000000..9451ff2 Binary files /dev/null and b/lectures/2021/16-data-flow-analysis/CS4200-2021-16-data-flow-analysis/CS4200-2021-16-data-flow-analysis.031.png differ diff --git a/lectures/2021/17-conclusion/CS4200-2022-17-code-generation-mechanics.pdf b/lectures/2021/17-conclusion/CS4200-2022-17-code-generation-mechanics.pdf new file mode 100644 index 0000000..9e0e153 Binary files /dev/null and b/lectures/2021/17-conclusion/CS4200-2022-17-code-generation-mechanics.pdf differ diff --git a/lectures/2021/17-conclusion/CS4200-2022-17-code-generation-mechanics/CS4200-2022-17-code-generation-mechanics.001.png b/lectures/2021/17-conclusion/CS4200-2022-17-code-generation-mechanics/CS4200-2022-17-code-generation-mechanics.001.png new file mode 100644 index 0000000..510ba98 Binary files /dev/null and b/lectures/2021/17-conclusion/CS4200-2022-17-code-generation-mechanics/CS4200-2022-17-code-generation-mechanics.001.png differ diff --git a/lectures/2021/17-conclusion/CS4200-2022-17-conclusion.pdf b/lectures/2021/17-conclusion/CS4200-2022-17-conclusion.pdf new file mode 100644 index 0000000..8916195 Binary files /dev/null and b/lectures/2021/17-conclusion/CS4200-2022-17-conclusion.pdf differ diff --git a/lectures/index.html b/lectures/index.html new file mode 100644 index 0000000..6cb0f33 --- /dev/null +++ b/lectures/index.html @@ -0,0 +1,790 @@ + + + + + + + + + Lectures + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+

Lectures

+
+ See also: + Homework Assignments + | Project +
+
+
+Note that the lecture schedule is tentative. For your convenience, last year's lectures are listed marked with (2020) in the title. These will be updated to their definitive 2021 versions. In particular the schedule for the CS4200-B part starting at Lecture 10 is expected to change. +
+ + + + + + + + +
+
+
+

2021

+
+
+
+
+ +
+
+
+

2022

+
+
+
+
+ + + + + +
+ +
+ + + + + + + + + + + diff --git a/milestone/0.html b/milestone/0.html new file mode 100644 index 0000000..27e45b7 --- /dev/null +++ b/milestone/0.html @@ -0,0 +1,366 @@ + + + + + + + + + Milestone 0: Test Your GitLab Set-Up + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Milestone 0: Test Your GitLab Set-Up + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 10, 2021 +
+
+
+
+ +
+
+

In this milestone we test that your GitLab set-up works.

+ +

Objective

+ +

Testing that your GitLab repository works with our automatic grading set-up.

+ +

Your repository should consist of (at least) the following:

+ +
    +
  1. A Spoofax project located in either the chocopy.syntax (preferred) or chocopy subfolder of your repository
    2. +
  2. The project should contain at least one example.
    3. +
  3. The project should contain at least one test.
    4. +
  4. Your language project builds.
    5. +
  5. Your language project parses a program consisting of just 1.
    6. +
+ +

Submission

+ +

You need to submit your ChocoPy project with a merge request on GitLab. +The Git and submission documentation explains how to file such a request.

+ +

The deadline for submission is September 10 2021, 23:59.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/milestone/1.html b/milestone/1.html new file mode 100644 index 0000000..aaa5c30 --- /dev/null +++ b/milestone/1.html @@ -0,0 +1,400 @@ + + + + + + + + + Milestone 1: Syntax Complete + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Milestone 1: Syntax Complete + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ September 27, 2021 +
+
+
+
+ +
+
+

In this milestone you submit your syntax definition.

+ +

Submission

+ +

You need to submit your ChocoPy project with a merge request on GitLab. +The Git and Submissions documentation explains how to file such a request.

+ +

The deadline for submission is September 26 2021, 23:59.

+ +

Grading

+ +

We grade your submission based on compliance with the ChocoPy syntax as defined in the reference manual, taking into account:

+ +
    +
  • lexical syntax
    • +
  • context-free syntax
    • +
  • disambiguation +
      +
    • associativity
      • +
    • precedence
      • +
    +
    • +
  • layout constraints
    • +
+ +

Additionally, we grade your submission manually to confirm that your SDF3 grammar is idiomatic, organized, and follows best practices. To determine whether your grammar is idiomatic, we use the following guidelines:

+ +
    +
  • Your grammar definition is organized. +
      +
    • Formatting is consistent across sections/files.
      • +
    • Grammars are split up into multiple files where appropriate.
      • +
    +
    • +
  • Your grammar does not use deprecated or low-level SDF3 features. +
      +
    • You do not use deprecated attributes such as {prefer}.
      • +
    • You do not use bare “kernel” syntax (syntax sections that are not explicitly lexical or context-free).
      • +
    +
    • +
  • Your grammar uses template syntax (<>) to enable pretty-printing.
    • +
  • Your grammar uses layout constraints to parse ChocoPy. +
      +
    • Your grammar does not use lexical elements for layout, such as NEWLINE, INDENT or DEDENT.
      • +
    • Your grammar uses high-level layout constraints (single-line, newline-indent, align-list, …) where possible.
      • +
    +
    • +
  • Your grammar uses priorities and follow restrictions to disambiguate. +
      +
    • Your grammar does not use separate context-free sorts for the sole purpose of disambiguating.
      • +
    +
    • +
  • Your grammar is as “flat” as reasonably possible. +
      +
    • Your grammar uses new sorts primarily for optionals, variants, or to extract out common patterns.
      • +
    • Your grammar generally uses a single sort for expressions and statements. +
        +
      • You do not have separate CExpr and Expr sorts.
        • +
      +
      • +
    • Note: This rule is a guideline. Do not feel forced to needlessly flatten sorts if you believe introducing new ones will aid readability.
      • +
    +
    • +
+ +

The bolded section is worth repeating. The ChocoPy reference manual uses two separate expression sorts in order to disambiguate and keep compatibility with Python grammar. SDF3 is powerful enough to represent these restrictions without the need for two separate sorts. In order to keep the SDF3 grammar as idiomatic as possible, as well as aid future static analysis steps, we therefore expect you to implement expressions using a single expression sort. For some more hints on how to do this, see also the (updated) lab 2 guide.

+ +

Your parser will automatically be graded in GitLab using SPT tests. These automated tests determine 70% of your grade for milestone 1, with the remaining 30% graded based on whether or not your grammar is idiomatic. A solution that passes all SPT tests but does not adhere to any of the above guidelines will therefore yield a 7 as final grade, whereas a less-complete implementation that follows all style guidelines closely will likely score better.

+ +

Early Feedback

+ +

We provide early feedback for the correctness of your syntax definition. This feedback gives you an indication which parts of your syntax definition might still be wrong. It includes a summary on how many tests you pass and how many points you earn by passing them. You have 15 early feedback attempts. Afterwards, you may still submit new merge requests, but you will not be able to receive feedback on them. Your final grade is based on the last submission done to the milestone-1-submission branch.

+ + + + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/milestone/2.html b/milestone/2.html new file mode 100644 index 0000000..c4d09a5 --- /dev/null +++ b/milestone/2.html @@ -0,0 +1,402 @@ + + + + + + + + + Milestone 2: Front-End Complete + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Milestone 2: Front-End Complete + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ October 24, 2021 +
+
+
+
+ +
+
+

In this milestone you re-submit your syntax definition and submit your static semantic definition.

+ +

Submission

+ +

You need to submit your ChocoPy project with a merge request on GitLab. +The Git and Submissions documentation explains how to file such a request.

+ +

The deadline for submission is October 24 2021, 23:59.

+ +

Grading

+ +

We grade your submission based on compliance with the ChocoPy syntax and static semantics as defined in the reference manual. During the course of the project, you therefore get another chance to improve your score for your syntax definition.

+ +

Your parser and static semantics should be defined in the same Spoofax project. The grader will search for your language definition in the chocopy.types, chocopy, and chocopy.syntax folders (in-order), using the first valid project it finds.

+ +

Early Feedback

+ +

We provide early feedback for the correctness of your syntax definition. +This feedback gives you an indication which parts of your syntax definition might still be wrong. +It includes a summary of how many tests you pass and how many points you earn by passing them. +You have 15 early feedback attempts. Please make your submissions to the milestone-2-submission branch.

+ +

We also provide you with the opportunity to improve your Syntax Definition once again, by providing Early Feedback for the milestone-1-feedback branch again. You can choose to improve your Syntax Definition before the deadline of milestone 2, as we will include it in the grading for milestone 2. If you choose to not make a new submission, the current state of the branch will be used for the grading process. You also have 15 Early Feedback attempts for this part of the assignment.

+ +

Use the latest Spoofax

+ +

Please make sure that your project works on the latest Spoofax version (Spoofax 3 releases). The grader will be using the most recent version to grade your project.

+ +

General development procedures

+ +

Here are some general tips you can apply when creating your static semantics definition:

+ +
    +
  • Do the homework exercises in WebLab and go through the slides and relevant literature, as they serve as an introduction to Statix.
    • +
  • Read the ChocoPy reference thoroughly. It helps to revisit it later to discover some edge cases.
    • +
  • Make use of the compiler on the ChocoPy website. If it compiles, it is valid semantics even if the program itself can crash (for example: print(None) is valid according to the type checker, but trying to execute this program will still result in an error).
    • +
  • Write tests! We expect you to write tests for valid and invalid static semantics, as well as name resolution and type checking. A more detailed tutorial on how to write tests for your static semantics can be found in Lab 3B.
    • +
  • Start with small examples, and work your way up.
    • +
  • Make sure to assign proper ref and type properties to your AST nodes (See homework/slides).
    • +
+ +

Debugging Statix

+ +

Here are some things you can try when you are witnessing unexpected behavior in your editor:

+ +
    +
  • Temporarily comment out irrelevant rules/lines in Statix in order to narrow down the problem.
    • +
  • Write test cases to reproduce the problem you are facing
    • +
  • You can force the failure of a rule by using false as a constraint. For example: programOk(_) :- false.
    • +
  • You can (temporarily) log some information or variables by using the try { <constraint> } | <error> <message> <location> constraint. For example: programOk(program) :- try { false } | note $[Error message for [program]]@program. . Here <error> takes the form of note, warning or error (which will cause the analysis to fail), and <location> is optional.
    • +
  • Draw out scope graphs on paper. Are there differences between the scope graph that is defined by your rules, and the one you expect?
    • +
+ +

Static Semantics

+ +

We expect your static semantic definition to perform valid name resolution and type checking. Some things to pay extra attention to are:

+ +
    +
  • Inheritance
    • +
  • Nested functions
    • +
  • Global/nonlocal declarations
    • +
  • Shadowing variables, fields and function names
    • +
  • Built-in/native functions and constructors
    • +
  • List types (least upper bound)
    • +
  • Return Statements
    • +
  • Method overriding
    • +
  • Return type of void functions
    • +
  • Usage/resolution before the definition
    • +
+ +

A small note on types

+ +

It is highly recommended you directly use the following types in your Statix rules:

+ +
    +
  • Integer: Int()
    • +
  • Boolean: Bool()
    • +
  • String: String()
    • +
  • List of Type T: List(T)
    • +
  • Empty list type: EmptyList()
    • +
  • None type: NoneType()
    • +
  • Class with name Foo and scope s: ClassType(s, "Foo")
    • +
  • Function with return Type RT and a list of parameter types PTs: FunType(RT, PTs)
    • +
  • Object: Object()
    • +
+ +

These are the types we test for in our early feedback/grading pipeline.

+ +

If you use different types in your Statix rules, you need to transform (i.e. map) your types to our types. For this purpose, transformation rules in the Stratego language are given in Lab 3B, which you have to extend/implement/test yourself to make them suitable for your type checker.

+ +

Write your own tests for type checking to confirm that your transformation rules work. Please include these rules in your submission, as we need them in our tests. Make sure to define a sort and the different constructors of your custom types in this file as well, so Stratego can recognize them, and match on them.

+ +

When all else fails…

+ +

Should you run into any issues regarding Statix or the assignment, please contact us during the labs (Tuesday Lab or Wednesday Shared Study Lab) or on Mattermost.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/milestone/3.html b/milestone/3.html new file mode 100644 index 0000000..028e377 --- /dev/null +++ b/milestone/3.html @@ -0,0 +1,306 @@ + + + + + + + + + Milestone 3: Back-End Complete + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + +
+
+ + +
+
+

+ + Milestone 3: Back-End Complete + +

+
+
+ + +
+
+ + + Project + + + + + + + +
+
+ January 21, 2022 +
+
+
+
+ +
+
+

In this milestone you submit the code of your compiler back-end, including a short report documenting its design.

+ +

Submission

+ +

You need to submit your ChocoPy project with a merge request on GitLab. +The Git and Submissions documentation explains how to file such a request.

+ +

Deadline

+ +

The deadline for submission is January 21, 2022, 23:59.

+ +

Report

+ +

In your submission, include a short report in PDF format. The report (a free-form document, that does not have to be very long) should document what choices you have made in your implementation, and what transformation mechanisms you have used. +Include at least descriptions of your approach to storage of intermediate values (registers or stack, register allocation), control-flow statements, calling convention, nested functions, translation of classes, and method invocation. +If you have addressed any challenges, please make sure to note that in the report and explain your approach.

+ +

In your report use examples to illustrate your compilation scheme. That is, small ChocoPy programs (or program fragments) and the RISC-V code they are translated to by your compiler.

+ +

Grading

+ +

We grade your submission based on compliance with the ChocoPy dynamic semantics as defined in the reference manual.

+ +

Grading Components

+ +

The grade consists of two main components for which you get a grade on a 0 - 10 scale. The components are weighted as follows.

+ +
    +
  • Compliance (weight: 0.8): To what extent does your back-end correctly implement the ChocoPy dynamic semantics? Compliance will be graded automatically based on a series of test programs and based on your report. Points are distributed as follows: +
      +
    • expressions: 2
      • +
    • statements: 2
      • +
    • basic functions: 1.5
      • +
    • nested functions: 1.5
      • +
    • objects: 2
      • +
    • lists: 1
      • +
    +
    • +
  • Challenges (weight: 0.2): Address any challenge in the implementation and describe that in your report, illustrated with examples. The grade will weigh the difficulty of the challenge and the quality of the solution. The grade will be based on your description in the report and tests that you provide in your repository. Possible challenges to address: +
      +
    • register allocation
      • +
    • efficient register-based calling convention
      • +
    • memory management
      • +
    • optimization
      • +
    +
    • +
+ +

Early Feedback

+ +

We provide early feedback for the correctness of your back-end. +This feedback gives you an indication which parts of your implementation might still be wrong. +It includes a summary of how many tests you pass and how many points you earn by passing them. +You have 30 early feedback attempts. +Please make your submissions to the milestone-3-submission branch.

+ + +
+
+
+
+ +
+ + +
+
+
+ +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + diff --git a/news/index.html b/news/index.html new file mode 100644 index 0000000..305f22f --- /dev/null +++ b/news/index.html @@ -0,0 +1,303 @@ + + + + + + + + + News + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+

Upcoming Events

+
+ + + +
+
+ > Jan 18, 2022 +
+
+
    + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ +
+

News Archive

+
+ + + + + + + +
+
+
+

2021

+
+
+
+
    + + +
  • +
    +

    Migrated website to 2021-2021 academic year.

    + + August 27, 2021 + + +
    +
    • + + + + + +
+ +
+
+
+

2020

+
+
+
+
    + + +
  • +
    +

    Homework and project assignments about type constraints available

    + + September 21, 2020 + + +
    +
    • + + + + +
  • +
    +

    Homework assignments about disambiguation available

    + + September 14, 2020 + + +
    +
    • + + + + +
  • +
    +

    The recording of today's lecture is available on Brightspace.

    + + September 03, 2020 + + +
    +
    • + + + + +
  • +
    +

    The website of the course in 2019 is at tudelft-cs4200-2019.github.io

    + + August 31, 2020 + + +
    +
    • + + + + +
  • +
    +

    A new website for the compiler construction course

    + + August 31, 2020 + + +
    +
    • + + + + +
  • +
    +

    Introduction lecture What is a compiler? on Thursday, September 3, 10:45 - 12:45. Online venue will be announced on Brightspace.

    + + August 31, 2020 + + (event on September 03, 2020) + + +
    +
    • + +
+ + + + +
+ +
+ + + + + + + + + + + diff --git a/project/2021/00-lab0a/chocopy-refman.png b/project/2021/00-lab0a/chocopy-refman.png new file mode 100644 index 0000000..3391493 Binary files /dev/null and b/project/2021/00-lab0a/chocopy-refman.png differ diff --git a/project/2021/00-lab0b/netid.png b/project/2021/00-lab0b/netid.png new file mode 100644 index 0000000..e5293b5 Binary files /dev/null and b/project/2021/00-lab0b/netid.png differ diff --git a/project/2021/00-lab1a/spoofax.png b/project/2021/00-lab1a/spoofax.png new file mode 100644 index 0000000..85999c6 Binary files /dev/null and b/project/2021/00-lab1a/spoofax.png differ diff --git a/project/2021/00-lab1b/spt.png b/project/2021/00-lab1b/spt.png new file mode 100644 index 0000000..b3698fe Binary files /dev/null and b/project/2021/00-lab1b/spt.png differ diff --git a/project/2021/00-lab1c/grammar.png b/project/2021/00-lab1c/grammar.png new file mode 100644 index 0000000..cbf3047 Binary files /dev/null and b/project/2021/00-lab1c/grammar.png differ diff --git a/project/2021/00-submissions/clone_url.png b/project/2021/00-submissions/clone_url.png new file mode 100644 index 0000000..f88620c Binary files /dev/null and b/project/2021/00-submissions/clone_url.png differ diff --git a/project/2021/00-submissions/merge_request.png b/project/2021/00-submissions/merge_request.png new file mode 100644 index 0000000..65f8de7 Binary files /dev/null and b/project/2021/00-submissions/merge_request.png differ diff --git a/project/2021/00-submissions/new_merge_request.png b/project/2021/00-submissions/new_merge_request.png new file mode 100644 index 0000000..6ac2f71 Binary files /dev/null and b/project/2021/00-submissions/new_merge_request.png differ diff --git a/project/index.html b/project/index.html new file mode 100644 index 0000000..8047786 --- /dev/null +++ b/project/index.html @@ -0,0 +1,993 @@ + + + + + + + + + Course Project: A Compiler for ChocoPy + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+

Course Project: A Compiler for ChocoPy

+
+ See also: + Lectures + | Project +
+
+Note that we list last year's lab descriptions marked 2020. They will be updated to their definitive version for this year. In particular, the CS4200-B part, starting at Lab 8, is expected to change. Check with the teaching staff before you work ahead. +
+ + + + + + + + +
+
+
+

2021

+
+
+
+
+ +
+
+
+

2022

+
+
+
+
+ + + + + +
+ +
+ + + + + + + + + + + diff --git a/publications/2010/KatsV10.pdf b/publications/2010/KatsV10.pdf new file mode 100644 index 0000000..9feeb03 Binary files /dev/null and b/publications/2010/KatsV10.pdf differ diff --git a/publications/2011/KatsVV11.pdf b/publications/2011/KatsVV11.pdf new file mode 100644 index 0000000..e7e2ba5 Binary files /dev/null and b/publications/2011/KatsVV11.pdf differ diff --git a/publications/2012/VollebregtKV12.pdf b/publications/2012/VollebregtKV12.pdf new file mode 100644 index 0000000..59e7be7 Binary files /dev/null and b/publications/2012/VollebregtKV12.pdf differ diff --git a/publications/2015/NeronTVW15.pdf b/publications/2015/NeronTVW15.pdf new file mode 100644 index 0000000..0bbc8b9 Binary files /dev/null and b/publications/2015/NeronTVW15.pdf differ diff --git a/publications/2016/AntwerpenNTVW16.pdf b/publications/2016/AntwerpenNTVW16.pdf new file mode 100644 index 0000000..d300989 Binary files /dev/null and b/publications/2016/AntwerpenNTVW16.pdf differ diff --git a/publications/2018/AntwerpenPRV18.pdf b/publications/2018/AntwerpenPRV18.pdf new file mode 100644 index 0000000..6667682 Binary files /dev/null and b/publications/2018/AntwerpenPRV18.pdf differ diff --git a/publications/2020/BravenboerDOV06.pdf b/publications/2020/BravenboerDOV06.pdf new file mode 100644 index 0000000..2b24880 Binary files /dev/null and b/publications/2020/BravenboerDOV06.pdf differ diff --git a/publications/2020/RouvoetAPKV20.pdf b/publications/2020/RouvoetAPKV20.pdf new file mode 100644 index 0000000..f78fd57 Binary files /dev/null and b/publications/2020/RouvoetAPKV20.pdf differ