eelco-visser-compiler-const.../lab/3b.html

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <meta name="description" content="">
    <meta name="author" content="Eelco Visser">
    <meta name="generator" content="Jekyll v3.8.5">
    <title>Lab 3(b): Creating Static Analysis Tests</title>
    <!-- <base href="/2021"> -->

    <!--link rel="canonical" href="https://getbootstrap.com/docs/4.3/examples/starter-template/"-->

		<link rel="icon" href="../img/logo/pl_ico2_2B3_icon.ico" type="image/x-icon">

    <!-- Bootstrap core CSS -->
<!--link href="https://getbootstrap.com/docs/4.3/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous"-->

<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">


    <style>
      .bd-placeholder-img {
        font-size: 1.125rem;
        text-anchor: middle;
        -webkit-user-select: none;
        -moz-user-select: none;
        -ms-user-select: none;
        user-select: none;
      }

      @media (min-width: 768px) {
        .bd-placeholder-img-lg {
          font-size: 3.5rem;
        }
      }
    </style>
    <!-- Custom styles for this template -->
    <link href="../css/main.css" rel="stylesheet">
    <link href="../css/borders-responsive.css" rel="stylesheet">

    <link rel="stylesheet" href="../css/pl.css">
  </head>
<body>

<nav class="navbar navbar-expand-md navbar-dark bg-dark fixed-top">

	<a class="navbar-brand" href="../index.html">
		TU Delft | CS4200
	</a>

  <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarsExampleDefault" aria-controls="navbarsExampleDefault" aria-expanded="false" aria-label="Toggle navigation">
    <span class="navbar-toggler-icon"></span>
  </button>

  <div class="collapse navbar-collapse" id="navbarsExampleDefault">
    <ul class="navbar-nav mr-auto">


        <li class="nav-item active">
          <a class="nav-link" href="../lectures/index.html" tabindex="-1" aria-disabled="true">Lectures</a>
        </li>


        <li class="nav-item active">
          <a class="nav-link" href="../homework/index.html" tabindex="-1" aria-disabled="true">Homework</a>
        </li>


        <li class="nav-item active">
          <a class="nav-link" href="../project/index.html" tabindex="-1" aria-disabled="true">Project</a>
        </li>


        <li class="nav-item active">
          <a class="nav-link" href="../news/index.html" tabindex="-1" aria-disabled="true">News</a>
        </li>


        <li class="nav-item active">
          <a class="nav-link" href="../blog/index.html" tabindex="-1" aria-disabled="true">Blog</a>
        </li>


    </ul>

  </div>
</nav>


<div class="container">


<div class="row">
		<div class="col-sm-12 col-md-12 col-lg-12 col-xl-12">


			<div class="mt-3 mb-3 pt-3 pb-3 text-dark border-top border-bottom border-grey">
				<div class="text-dark font-weight-bold">
					<h1>

					  Lab 3(b): Creating Static Analysis Tests

					</h1>
				</div>
				<div>


			  </div>
			  <div>


								Project


			  </div>
			  <div>
					September 22, 2021
				</div>
      </div>
  </div>
</div>

<div class="row">
	<div class="col-sm-12 col-md-12 col-lg-9 border-lg-right border-grey">
		<p>Before getting started with your static analysis definition, you set up a test suite for static analysis. Develop the test suite in tandem with the development of your static analysis. The test suite consists of positive and negative test cases. We will not grade this test suite, but you should develop one to get confidence in the quality of your static analysis. When you are asking for help from the course staff, we will first ask what tests you have written to demonstrate the problem.</p>

<h3 id="objectives">Objectives</h3>

<p>In your chocopy project, develop a test suite for static analysis. The test suite should provide</p>

<ol>
  <li>Test cases for types</li>
  <li>Test cases for name resolution</li>
  <li>Test cases for errors</li>
</ol>

<h3 id="testing-type-constraints">Testing Type Constraints</h3>

<p>See the example tests in the WebLab homework assignments for Week 3.</p>

<h3 id="testing-reference-resolution">Testing Reference Resolution</h3>

<p>In test cases for reference resolution,
 you write syntactically correct programs and
 mark names at definition and use sites with inner square bracket blocks.
You can then relate the use site with the definition site in a <code class="language-plaintext highlighter-rouge">resolve x to y</code> clause,
  using numbers to refer to the inner blocks.
For example, the following two test cases require to resolve the type <code class="language-plaintext highlighter-rouge">Foo</code> to the name in the definition of class <code class="language-plaintext highlighter-rouge">Foo</code>:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>module resolution

test class name resolution[[
class [[A]](object):
	pass

class B(object):
	a:[[A]] = None
]] resolve #2 to #1

test class field resolution[[
class Foo(object):
	[[bar]]:int = 0

foo:Foo = None
foo = Foo()
print(foo.[[bar]])
]] resolve #2 to #1
</code></pre></div></div>

<p>After copying this into an SPT file Spoofax will add the error <em>“Reference resolution failed”</em> and <em>“No constraint generation rule for …“</em>. This is expected, since your project is missing an implementation for reference resolution (this is part of the next lab).</p>

<p>You can use <em>fixtures</em> to avoid repeating parts in similar test cases. See the SPT documentation for details.</p>

<p>You should come up with test cases for the resolution of class names, field names, parameter names, and variable names.
Start with simple test cases, but keep in mind that coverage is the main criterion for your grade.
It is important to think about
 forward and backward references,
 global and non-local declarations in functions,
 and resolution in the presence of homonyms.</p>

<p class="notice notice-warning">Make sure that there are no errors in tests with a <code class="language-plaintext highlighter-rouge">resolve x to y</code> clause, these tests are invalid when there are errors.</p>

<h3 id="testing-error-checking">Testing Error Checking</h3>

<p>In test cases for error checking, you need to specify the number of errors, warnings, or notes in a
test case in <code class="language-plaintext highlighter-rouge">errors</code>, <code class="language-plaintext highlighter-rouge">warnings</code>, or <code class="language-plaintext highlighter-rouge">notes</code> clauses.  For example, the following test cases
specify a correct ChocoPy program, a program with two errors which are reported on the name of a
duplicate class <code class="language-plaintext highlighter-rouge">Foo</code>, and another program with an error which is reported on the name of an unknown
class <code class="language-plaintext highlighter-rouge">Bar</code>:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>module correctness

language chocopy

test correct program [[
class Counter(object):
	count:int = 0

	def getCount(self:Counter)-&gt;int:
		return self.count

	def increaseCount(self:Counter):
		self.count = self.count + 1
]] 0 errors

test incorrect program [[
class Counter(object):
	count:bool = False

	def getCount(self:Counter)-&gt;int:
		return self.count

	def increaseCount(self:Counter):
		self.count = self.count + 1
]] &gt;= 1 errors // or 3 errors

test error on unknown class [[
class Foo(object):
	bar:Bar = None
]] &gt;= 1 errors
</code></pre></div></div>

<p>You can start with test cases for duplicate and missing definitions.  Similar to your syntax test
cases, you can pair up positive (<code class="language-plaintext highlighter-rouge">0 errors</code>) and negative test cases.  For duplicate definitions, we
expect errors on the definitions with the same name.</p>

<p class="notice notice-warning">The number of errors can be hard to predict, because errors sometimes cascade. Therefore, if you
expect any errors, you should use the <code class="language-plaintext highlighter-rouge">&gt;= 1 errors</code> expectation, even if you expect a specific
number of errors. For example, this expectation was used in the duplicate class test, even though we
would expect exactly two errors.</p>

<p>Next, you should develop test cases for fields and variables which hide global variables, global and non-local declarations, and class
instantiation, subclassing, referencing.  Again, you should keep in mind that coverage is the main
criterion for your grade.</p>

<h3 id="testing-types-of-expressions">Testing Types of Expressions</h3>

<p>In test cases for type analysis,
 you write syntactically correct programs and
 mark expressions with inner square bracket blocks.
You can then specify the expected type of the marked expression in a <code class="language-plaintext highlighter-rouge">run x to y</code> clause.
For example, the following two test cases require an integer literal to be of type <code class="language-plaintext highlighter-rouge">Int()</code>
and a variable reference to be of its declared type <code class="language-plaintext highlighter-rouge">Bool()</code>:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>module types

test integer literal [[
print([[1]])
]] run get-type on #1 to Int()

test boolean condition [[
b1:bool = True
b2:bool = False
if [[b1 and b2]]:
	print("Yes!")
]] run get-type	on #1 to Bool()
</code></pre></div></div>

<p>In order for these tests to succeed, you need to define your own Stratego rule <code class="language-plaintext highlighter-rouge">get-type</code>. For this, you can copy and paste the following code into <code class="language-plaintext highlighter-rouge">src/main.str2</code></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>signature
  sorts
    Type

  constructors
    Int : Type
    Bool : Type
    String : Type
    ClassType : scope * ID -&gt; Type
    List : Type -&gt; Type
    NoneType : Type
    EmptyList : Type
    Object : Type
    FunType : Type * list(Type) -&gt; Type
    // We expect to get these types in the grading pipeline
    // you are free to use your own (custom) types in your Statix file
    // as long as you write transformation rules from your types to our types
    // in resolve-type

    // Here you can define the signature of the types you have used in your Statix definitions.
    // This is just an example on how to do it.
    // signature
    //   sorts
    //     MyCustomType
    //
    //  constructors
    //    MyCustomIntType : MyCustomType
    //    MyCustomClassType : ID * scope -&gt; MyCustomType

rules
  // get-type: SimpleStatement(expr) -&gt; &lt;get-type&gt; expr // An example on how to match on AST nodes.
  get-type: expr -&gt; type' // Defines a rule which matches on node and returns type'
    where
      // Assigns variable a to be the result of the Statix analysis of the entire program (or throws an error)
      a := &lt;stx-get-ast-analysis &lt;+ fail-msg(|$[no analysis on node [&lt;strip-annos;write-to-string&gt; expr]])&gt;;
      // Gets the type of the given node (or throws an error)
      type := &lt;stx-get-ast-type(|a) &lt;+ fail-msg(|$[no type on node [&lt;strip-annos;write-to-string&gt; expr]])&gt; expr;
      // Calls a rule to convert the type given by Statix to a type for our tests.
      type' := &lt;resolve-type&gt; type

  // resolve-type: MyCustomIntType() -&gt; Int() // This rule matches on your custom type, and returns a type we need in our tests.
  // resolve-type: MyCustomClassType(name, scope) -&gt; ClassType(scope, name) // This rule matches on your custom type, and returns a type we need in our tests.
  // TODO: Define your own transformation rules to transform types from your Statix defintion to the types we expect in our tests
  resolve-type: a -&gt; a // This rule matches on a and returns a (trivial rule given as example). This implies no transformation actually takes place. Define your own rules ABOVE this rule.

  fail-msg(|msg) = err-msg(|$[get-type: [msg]]); fail
</code></pre></div></div>

<p>This code defines a Stratego rule <code class="language-plaintext highlighter-rouge">get-type</code> which we can use to map AST Nodes to their types using the results of the Statix analysis. In our tests for the Early Feedback and Grading, we expect certain types, so it is highly recommended to use the same types in your Statix rules. If not, you might need to perform some extra transformations from your types to our types in the <code class="language-plaintext highlighter-rouge">resolve-type</code> rule.</p>

<p>For more information on the Stratego language, visit the Stratego documentation.</p>

<p>You can use <em>fixtures</em> to avoid repeating parts in similar test cases. See the
SPT documentation
for details.</p>

<p>When applying <code class="language-plaintext highlighter-rouge">get-type</code> to objects, we expect a <code class="language-plaintext highlighter-rouge">ClassType</code> constructor.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>test class type [[
class Foo(object):
	pass
[[Foo()]]
]] run get-type on #1 to ClassType(_, "Foo") // We don't care about the scope, only the classname.
</code></pre></div></div>

<p>You should come up with test cases for the types of all kinds of expressions. Just like previous
testing assignments, this assignment is all about the coverage of your test suite.</p>

<p>The constructors for various types are:</p>

<ul>
  <li>Integer: <code class="language-plaintext highlighter-rouge">Int()</code></li>
  <li>Boolean: <code class="language-plaintext highlighter-rouge">Bool()</code></li>
  <li>String: <code class="language-plaintext highlighter-rouge">String()</code></li>
  <li>List of Type T: <code class="language-plaintext highlighter-rouge">List(T)</code></li>
  <li>Empty list type: <code class="language-plaintext highlighter-rouge">EmptyList()</code></li>
  <li>None type: <code class="language-plaintext highlighter-rouge">NoneType()</code></li>
  <li>Class with name Foo and scope s: <code class="language-plaintext highlighter-rouge">ClassType(s, "Foo")</code></li>
  <li>Function with return Type RT and a list of parameter types PTs: <code class="language-plaintext highlighter-rouge">FunType(RT, PTs)</code></li>
  <li>Object: <code class="language-plaintext highlighter-rouge">Object()</code></li>
</ul>

<p class="notice notice-warning">Tests that contain a <code class="language-plaintext highlighter-rouge">run x to y</code> clause are invalid if the semantic analysis yields errors. Ensure that the snippet of ChocoPy that you are testing does not contain any errors if you want to use <code class="language-plaintext highlighter-rouge">get-type</code> or other strategies.</p>

<p class="notice notice-warning">If you use a start symbol other than <code class="language-plaintext highlighter-rouge">Program</code>, replace any samples in this guide that use <code class="language-plaintext highlighter-rouge">Program</code> with your start symbol instead.</p>

<p>Make sure to actually append your AST nodes with their types using <code class="language-plaintext highlighter-rouge">@x.type := T</code> (in a similar way to assigning the <code class="language-plaintext highlighter-rouge">ref</code> property, which is needed for name resolution). Otherwise the rule will not find any types on your AST node.</p>

<h3 id="testing-method-name-resolution">Testing Method Name Resolution</h3>

<p>Consider the following test case as an example:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>test method name resolution [[
class Foo(object):
	def [[run]](self:Foo)-&gt;int:
		return 1
Foo().[[run]]()
]] resolve #2 to #1
</code></pre></div></div>

<p>The type of the callee expression determines the class in which the method declaration can be found.
In this example, the expression <code class="language-plaintext highlighter-rouge">Foo()</code> is of type <code class="language-plaintext highlighter-rouge">ClassType(_, "Foo")</code> and
the corresponding class <code class="language-plaintext highlighter-rouge">Foo</code> contains a method declaration for <code class="language-plaintext highlighter-rouge">run()</code>.</p>

<p>You should come up with test cases for the resolution of method names.
Start with simple test cases, but keep in mind that method name resolution is quite complex
 and that coverage is the main criterion for your grade.
It is important to think about forward and backward references,
 resolution in the presence of homonyms and overriding,
 and the influence of class hierarchies on resolution.</p>

<p>You should also come up with test cases for error checking on method names.
This should include test cases for errors on duplicate definitions, missing definitions, and method overloading.
Similar to previous test cases, you can pair up positive (<code class="language-plaintext highlighter-rouge">0 errors</code>) and negative test cases.</p>

<p class="notice notice-warning">Make sure that there are no errors in tests with a <code class="language-plaintext highlighter-rouge">resolve x to y</code> clause. These tests are invalid when there are errors.</p>

<h3 id="testing-type-error-checking">Testing Type Error Checking</h3>

<p>A type error occurs, when the type of an expression does not conform to its expected type.
Consider the following test case as an example:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>test print boolean [[
def printInt(i:int):
	print(i)
printInt(True)
]] 1 error
</code></pre></div></div>

<p>Type errors can occur in statements, expressions, and method declarations.
You should come up with test cases for such errors.
Subtyping is a common source for errors not only in programs, but also in language implementations.
It is therefore important to have positive and negative typing tests, which involve correct and incorrect subtyping.</p>

<p>Again, keep in mind that coverage is the main criterion for your grade.</p>

<h3 id="number-of-errors">Number of errors</h3>

<p>Similar to the previous testing lab, you need to be careful about the number of errors, because
errors sometimes cascade. For example, if you expect <em>2</em> errors, you should use the <code class="language-plaintext highlighter-rouge">&gt;= 2 errors</code>
expectation, even if you expect an exact number of errors.</p>

<h3 id="a-final-note">A final note</h3>
<p>A great way of testing expected ChocoPy behavior, when you want to know whether something is allowed or not, is to try it out on the <a href="https://chocopy.org/">ChocoPy Website</a></p>


  </div>
	<div class="col-sm-12 col-md-12 col-lg-3 col-xl-3 " >
		<div class="sticky-top top70 d-none d-lg-block d-xl-block">
		<div class="pb4 mb3 border-bottom border-grey ">
			<nav>
				<ul class="pagination justify-content-left">

						<li class="page-item">
							<a class="page-link" href="3a.html">
								&laquo;
							</a>
						</li>

					<li class="page-item">
						<a class="page-link" href="../project/index.html">
							^
						</a>
					</li>

						<li class="page-item">
							<a class="page-link"  href="3c.html">&raquo;</a>
						</li>

				</ul>
			</nav>
		</div>
		<ul id="my_toc" class="toc list-group list-group-flush d-none d-lg-block d-xl-block p-0 ml-0 mt-3">
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#objectives">Objectives</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#testing-type-constraints">Testing Type Constraints</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#testing-reference-resolution">Testing Reference Resolution</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#testing-error-checking">Testing Error Checking</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#testing-types-of-expressions">Testing Types of Expressions</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#testing-method-name-resolution">Testing Method Name Resolution</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#testing-type-error-checking">Testing Type Error Checking</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#number-of-errors">Number of errors</a></li>
  <li class="list-group-item pl-0 ml-0 border-0 pl-0 pt-0 pb-1 pr-0 m-0 mr-3"><a href="3b.html#a-final-note">A final note</a></li>
</ul>

		</div>
  </div>
</div>

<div class="row">
	<div class="col-12">
		<div class="border-top border-bottom border-grey mt-3 pt-3">
			<nav>
				<ul class="pagination justify-content-center">

						<li class="page-item">
							<a class="page-link" href="3a.html">
								Previous
							</a>
						</li>


						<li class="page-item">
							<a class="page-link"  href="3c.html">Next</a>
						</li>

					<li class="page-item">
						<a class="page-link" href="../project/index.html">
							Index
						</a>
					</li>
				</ul>
			</nav>
		</div>
  </div>
</div>

</div>


<!-- Optional JavaScript -->
 <!-- jQuery first, then Popper.js, then Bootstrap JS -->
 <script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
 <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js" integrity="sha384-UO2eT0CpHqdSJQ6hJty5KVphtPhzWj9WO1clHTMGa3JDZwrnQq4sF86dIHNDz0W1" crossorigin="anonymous"></script>
 <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js" integrity="sha384-JjSmVgyd0p3pXB1rRibZUAYoIIy6OrQ6VrjIEaFf/nJGzIxFDsf4x0xIM+B07jRM" crossorigin="anonymous"></script>

</body>
</html>