Worksheet Closures

Create a new SBT project so that you can easily compile Java and Scala source files.

Create a new directory called closures. It is recommended that you use a path without any spaces in it. For example, avoid C:\Users\Alice Smith\closures because the directory Alice Smith has a space in it. Spaces in path names can confuse some tools. The remaining instructions here will assume that your directory is in /tmp/closures; you should adjust the instructions for the location of your closures directory.

Create a file called /tmp/closures/build.sbt containing

name := "CSC447 Closures"
version := "1.0"
scalaVersion := "3.3.1"

scalacOptions ++= Seq(
  "-encoding", "UTF-8", // Character encoding used by source files.
  "-deprecation",       // Emit warning and location for usages of deprecated APIs.
  "-feature",           // Emit warning and location for usages of features that should be imported explicitly.
  "-unchecked",         // Enable additional warnings where generated code depends on assumptions.
  "-Yexplicit-nulls",   // http://dotty.epfl.ch/docs/reference/other-new-features/explicit-nulls.html
  "-new-syntax",        // Require `then` and `do` in control expressions.
  "-source:future",
  "-Xfatal-warnings",
)

javacOptions ++= Seq("-source", "17", "-target", "17")

Create a file called /tmp/closures/src/main/java/JHello.java (create the sequence of directories first, e.g., using mkdir -p /tmp/closures/src/main/java on Linux / OSX) containing

public class JHello {
  public static void main (String[] args) {
    System.out.println ("hello world");
  }
}

Create a file called /tmp/closures/src/main/scala/SHello.scala (create the sequence of directories first, e.g., using mkdir -p /tmp/closures/src/main/scala on Linux / OSX) containing

object SHello:
  def main (args:Array[String]) =
    println ("hello world")

In a command prompt / terminal with /tmp/closures/ as the current working directory, start SBT by running sbt. When the SBT prompt shows, enter compile at the SBT prompt. This will download files the first time it is run. Do not exit SBT because you will have to use it again, and it is slow to start.

At the SBT prompt enter run. You should see

> run

Multiple main classes detected, select one to run:

 [1] JHello
 [2] SHello

Enter number: 

Try entering 1 or 2 to run a program, then use run again with the other number.

Running programs from within SBT is very useful when the programs use libraries, because SBT includes all the libraries for you automatically.

You can run the Java program directly from the shell using

$ java -cp target/scala-*/classes JHello
hello world

where you replace * by the correct version number.

This and all subsequent examples are written for Linux/OS X. On Windows, you should use backslash in paths instead of slash, i.e., replace target/scala-*/classes with target\scala-*\classes.

If you have the scala command installed and in your PATH, you could run the Scala program using

$ scala -cp target/scala-*/classes SHello
hello world

Otherwise, start sbt again. At the SBT command prompt, enter stage. You should see something like

$ sbt
[info] Loading project definition from /tmp/closures/project
[info] Set current project to CSC447 Closures (in build file:/tmp/closures/)

> stage
[info] Packaging /tmp/closures/target/scala-*/csc347-closures_*-1.0-sources.jar ...
[info] Done packaging.
[info] Main Scala API documentation to /tmp/closures/target/scala-*/api...
[info] Wrote /tmp/closures/target/scala-*/csc347-closures_*.pom
[warn] Multiple main classes detected.  Run 'show discoveredMainClasses' to see the list
[info] Packaging /tmp/closures/target/scala-*/csc347-closures_*.jar ...
[info] Done packaging.
model contains 2 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /tmp/closures/target/scala-*/csc347-closures_*-javadoc.jar ...
[info] Done packaging.
[success] Total time: 5 s, completed Apr 2, 2018, 6:22:11 PM

That command creates a shell script for each main method in /tmp/target/universal/stage/bin. You can use that script to run Scala programs (on Linux and OS X, and Windows if you have Bash).

$ ./target/universal/stage/bin/j-hello
hello world

Examine the build directory /tmp/closures/target that was created when you compiled with SBT. You should have

ls target/scala-*/classes/
JHello.class  SHello.class  SHello$.class

Now you can run javap to show the fields and methods within each class file. The -p option includes private declarations that would not otherwise be displayed.

$ javap -p -cp target/scala-*/classes JHello
Compiled from "JHello.java"
public class JHello {
  public JHello();
  public static void main(java.lang.String[]);
}

$ javap -p -cp target/scala-*/classes SHello
Compiled from "SHello.scala"
public final class SHello {
  public static void main(java.lang.String[]);
}

$ javap -p -cp target/scala-*/classes SHello\$
Compiled from "SHello.scala"
public final class SHello$ {
  public static final SHello$ MODULE$;
  public static {};
  public void main(java.lang.String[]);
  private SHello$();
}

For Linux and OS X, you need to prevent the $ character being interpreted by the shell as the beginning of a shell variable (e.g., $PATH). As shown above, this can be done by using \$ to escape the $ character.

You can view the Java Virtual Machine (JVM) bytecode inside class files using the -c option. JVM bytecode is reminiscent of assembly language for many processors, e.g., x86 or ARM. Major differences include

1. The JVM is a stack-oriented machine, so, e.g., addition is performed by pushing two integers onto the stack, and then the iadd instruction is used. The addition pops the two integers, performs the addition, and pushes the result back onto the stack.
2. The bytecode for a method call is not able to modify anything on the call stack except in its own activation record. This provides some security guarantees that isolate untrusted code from trusted code. It is enforced statically by a bytecode-verification process that occurs when bytecode is loaded, i.e., it happens at runtime, but only when the code is initially loaded; it does not involve multiple runtime checks.

$ javap -c -p -cp target/scala-*/classes JHello
Compiled from "JHello.java"
public class JHello {
  public JHello();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public static void main(java.lang.String[]);
    Code:
       0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
       3: ldc           #3                  // String hello world
       5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
       8: return
}

Now try looking at the JVM bytecode for simple Java programs. Do this by writing new Java classes (saved in the same directory as JHello.java) and then running compile from SBT again. Alternatively, just run ~compile in SBT and it will keep recompiling your code every time you save Java or Scala files.

For example, look at the JVM bytecode for methods such as

void f () {
  int x = 0;
  while (x < 10) {
    x = x + 2;
  }
}

int fact (int x) {
  int result = 1;
  while (x >= 1) {
    result = result * x;
    x = x - 1;
  }
}

int factR (int x) {
  if (x == 1) {
    return 1;
  } else {
    return x * factR (x - 1);
  }
}

Aside: there are assemblers that take a text file containing JVM assembly language and then assemble it to class files. Related tools are used for optimization, code analysis, and program obfuscation (e.g., to make it difficult to reverse engineer a program).

The following Java program creates a thread that executes the run method of the Sender class. That is, if the initial thread for the program is t1, then it creates a second thread t2 that executes the run method of the Sender class. That method (running in thread t2) sends instances of PrintMessage back to the original thread (thread t1) using a BlockingQueue (a concurrent data structure with blocking put and take operations). Since PrintMessage also implements Runnable, thread t1 can execute the run method on them. They happen to print one String each time (although thread t1 has no knowledge or control over what they do).

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;


class PrintMessage implements Runnable {
  final String message;

  PrintMessage (String message) {
    this.message = message;
  }

  public void run () {
    System.out.println (message);
  }
}

class Sender implements Runnable {
  private final String[] messages;
  private final BlockingQueue<Runnable> q;

  Sender (String[] messages, BlockingQueue<Runnable> q) {
    this.messages = messages;
    this.q = q;
  }

  public void run () {
    try {
      System.out.println ("Sender started");
      for (int i = 0; i < messages.length; i++) {
        Thread.sleep (1000);
        q.put (new PrintMessage (messages[i]));
      }
    } catch (InterruptedException e) {
      // ignore interruptions
    }
  }
}

public class Threads1 {
  public static void main (String[] args) {
    try {
      String[] messages = new String[] { "the", "rain", "in", "Spain", "falls", "mainly", "on", "the", "plain" };
      BlockingQueue<Runnable> q = new LinkedBlockingQueue<> ();
      Sender sender = new Sender (messages, q);
      new Thread (sender).start ();
      while (true) {
        q.take ().run ();
      }
    } catch (InterruptedException e) {
      // ignore interruptions
    }
  }
}

Try compiling and running the Java program. You can use SBT to compile and run it (put the Java program in the same directory as JHello.java). Use control-C to kill the program, because thread t1 waits forever on the BlockingQueue.

Now rewrite the code so that PrintMessage and Sender are anonyomous inner classes inside the main method. Check that your code still compiles and runs.

Solution: Rewrite Non-Nested Classes Using Anonymous Inner Classes

Use javap to examine the class files that are produced when you compile your previous Java program using anonymous inner classes. How do the class files correspond to PrintMessage and Sender?

Try typing in and running the sequential map over a list from the lectures.

Then change the code so that it prints N x characters for each number N in the list. You should define a method that takes an Integer (or int) N and returns a String consisting of N x characters. You can then call that method from apply. You will need to update the type parameters to Function.

import java.util.*;
import java.util.function.Function;
import java.util.stream.*;


public class Nested {
  public static void main (String[] args) {
    List<Integer> l = new ArrayList<> ();
    Stream<Integer> s = l.stream();
    l.add (1); l.add (2); l.add (3);
    s.map (x -> x + 1)
     .collect (Collectors.toList ())
     .forEach (x -> System.out.format ("%2d ", x));
  }
}

// public class Nested {
//   public static void main (String[] args) {
//       List<Integer> l = Arrays.asList (new Integer[] {1,2,3});
//     l.add (1); l.add (2); l.add (3);
//     Function<Integer,Integer> f;
//     f = new Function<Integer,Integer> () { // anonymous inner class
//       public Integer apply (Integer x) { return x + 1; }
//     };
//     l.stream ().map (f)
//      .collect (Collectors.toList ())
//      .forEach (x -> System.out.println (x));
//   }
// }

Scala allows class declarations and object declarations. Classes, objects, and functions/methods can be nested arbitrarily.

Object declarations can be thought of as defining a new class and instantiating just one copy if (cf. the Singleton Pattern described in SE350/SE450). There are no static methods or classes.

Save the following as O.scala in the same directory as SHello.scala and compile it with SBT.

trait HasF { def f(): Unit }
object O:
  def main (args:Array[String]) =
    class C (x:Int):
      println ("C")
      object P extends HasF:
        println ("P")
        def f () = println (args (x))
    val cs:List[C] = for i <- (0 to (args.length - 1)).toList yield C(i)
    val ps:List[HasF] = for c <- cs yield c.P
    ps.foreach (p => p.f())

To run it, use the run command at the SBT prompt, but add command-line arguments after run. For example

> run the rain in spain

Multiple main classes detected, select one to run:

 [1] SHello
 [2] JHello
 [3] O

Enter number: 3

[info] Running O the rain in spain
the
rain
in
spain
[success] Total time: 1 s, completed Feb 18, 2016 11:47:40 AM

There is just one O at runtime.

How many instances of C are there at runtime (when run the command-line arguments above). Check your answer by adding println ("C") directly inside the class C body, so that C is printed each time a new C instance is created.

How many instances of P are there at runtime (when run the command-line arguments above). Check your answer by adding println ("P") directly inside the class P body, so that P is printed each time a new P instance is created.

When an object P is declared inside a class C, is there just one object P in the entire runtime, or is it one object P for each C instance? (The previous output answers this question).

Consider the following Scala code. In each of the functions foo1, …, foo7 there is a nested function (either explicitly named using def or an anonymous function).

Which of foo1, …, foo7 has a nested function whose lifetime extends beyond the lifetime of its enclosing function (foo1, …, foo7)? That is, which of the nested functions can run when the enclosing function (foo1, …, foo7) has returned?

object NestedFunc:
  def foo1 [X] (xs:List[X]) : List[X] =
    def aux (us:List[X], vs:List[X]) : List[X] =
      us match
        case Nil => vs
        case w::ws => aux (ws, w::vs)
    aux (xs, Nil)

  def foo2 [X] (xs:List[X], k:X=>X) : List[X] =
    def aux (us:List[X], vs:List[X]) : List[X] =
      us match
        case Nil => vs
        case w::ws => aux (ws, (k (w)) :: vs)
    aux (xs, Nil)

  def foo3 [X] (xs:List[X]) : Int=>List[X] =
    def aux (n:Int) : List[X] =
      if n <= 0 then
        Nil
      else
        xs ::: aux (n - 1)
    aux

  def foo4 [X] (xs:List[X]) : Unit =
    xs.foreach ((x:X) => println (x))

  def foo5 (xs:List[Int]) : Int =
    xs.foldLeft (0) ((x:Int,y:Int) => x + y)

  def foo6 (xs:List[Int]) : Int=>Int =
    (n:Int) => (xs.foldLeft (0) ((x:Int,y:Int) => x + y))

  var f : Int=>Int = x=>x
  def foo7 (x:Int) : Unit =
    f = ((y:Int) => x+y)

Solution: Recognize Lifetime of Nested Function

In the following output Threads1Solution$1 corresponds to Sender. It has the same fields. javac has not marked the fields as final, but they cannot be accessed easily from Java code because they have the $ character in their name. (They could be accessed via the Reflection API though).

Threads1Solution$1$1 corresponds to PrintMessage=. It has the same message field. In addition it has a reference to its enclosing object called this$0.

$ ls Threads1Solution*class
Threads1Solution$1$1.class  Threads1Solution$1.class  Threads1Solution.class

$ javap -p Threads1Solution
Compiled from "Threads1Solution.java"
public class Threads1Solution {
  public Threads1Solution();
  public static void main(java.lang.String[]);
}

$ javap -p Threads1Solution\$1
Compiled from "Threads1Solution.java"
final class Threads1Solution$1 implements java.lang.Runnable {
  final java.lang.String[] val$messages;
  final java.util.concurrent.BlockingQueue val$q;
  Threads1Solution$1(java.lang.String[], java.util.concurrent.BlockingQueue);
  public void run();
}

$ javap -p Threads1Solution\$1\$1
Compiled from "Threads1Solution.java"
class Threads1Solution$1$1 implements java.lang.Runnable {
  final java.lang.String val$message;
  final Threads1Solution$1 this$0;
  Threads1Solution$1$1(Threads1Solution$1, java.lang.String);
  public void run();
}

Each of foo3, foo6 and foo7 has a nested function that escapes the lifetime of the foo function. Both foo3 and foo6 return the nested function directly. foo7 stores the nested function in a non-local variable.

Each of the other foo functions defines a nested function, but the use of the nested function is contained within the lifetime of the call to foo.

In foo4 and foo5, the nested function is passed outside the scope of foo, but not the lifetime of foo: the lifetime of the receiving function (foreach and foldLeft, respectively) is contained within the lifetime of foo.