Diffuse JavaScript

Christian Queinnec
Professor emeritus Sorbonne University

This document is an HTML version of the slides accompanying the MOOC “Diffuse JavaScript”. It is made of sections that can be unfolded to reveal the titles of subsections that can be, in turn, unfolded to reveal subsubsections that is, the slides gathered by topics. Some subsubsections, marked with a grey triangle, are still folded since they hold finer details that can be safely omitted at first reading.

1  Before we start...

1.1  Preliminaries

1.1.1  Welcome

Welcome

Thanks for your registration and welcome to the “Diffuse JavaScript” MOOC.

I therefore assume that

  • You want to master JavaScript so you can use this skill to write diffuse applications that is, applications that run on both sides of the HTTP or WebSocket protocols.
  • You want to have a strong grip on events, callbacks, promises and generators that make the new version of JavaScript, known as ECMAscript 2015, so powerful.

All these reasons are excellent so welcome again to this MOOC!

Foreword

This course is not an initiation to programming, I expect that you know a little about programming (loops, functions, classes, methods, etc.). I also assume that you know the difference between a browser and a server, connected via GET or POST requests, according to the HTTP protocol, though this will only be useful after mid-course.

I will mainly write JavaScript code in ECMAscript 2015 (nicknamed ES6) and further revisions. I will not insist on the usual linguistic features of ES6 (those features that you can find in nearly every other programming language), I will not review the many usual methods but will focus and describe more thoroughly what makes ES6 so different.

Another important feature of that course is that we focus on the JavaScript language and its enormous potentialities that can be revealed and put to use rather simply. Therefore we will avoid using sophisticated frameworks that, though very useful, encumbers code with tons of extra conventions.

Outline

This MOOC is self paced but you may also practise it in four weeks:

  • The first two weeks are devoted to JavaScript, the language by itself.
  • Concurrency is the topic of the third week,
  • while distribution will be seen in the fourth and last week.

Caveats

Some caveats before we start:

  • There are so many features in JavaScript that forward references cannot be avoided!
  • Whole truth cannot be revealed immediately!
  • Ultimate truth does not fit in a four-week course!
  • JavaScript looks apparently simple but is full of oddities!

So, after this MOOC, you won’t know every corner detail but, you will master the main and most useful features of JavaScript and be able to write complex diffuse applications.

1.2  Requirements

1.2.1  Requirements

Node.js

For this course, in order to be able to run JavaScript code, you need to have Node.js available.

Instructions to install Node.js on your computer can be found on the Node.js site . Please take the most recent LTS 10.* version (LTS stands for Long Term Support) that implements ECMAscript 2015 and further features.

Installing Node.js also installs npm, the Node package manager, you will need it to adjoin useful libraries to JavaScript such as Jasmine a framework for unit testing. Install jasmine with


npm install jasmine

You may also use online JavaScript code editor such as jsfiddle or jsbin . However parts of the MOOC are specific to Node.js and require it to be run. In that case, you probably need an IDE .

Alternatively, you may also use the following Docker container that contains all the software required to solve all the proposed exercises.

1.3  Learning

1.3.1  Resources

How to

Learning alone is not easy and there is more than one way to learn. Lot of material is present in this MOOC to allow you to choose your own particular way of learning.

  • You may read the chopped inline HTML version displayed with OpenEdX,
  • Alternatively, you may prefer to read or print the PDF version or HTML version of the MOOC.
  • Examine code examples,
  • Examine the Jasmine tests to go deeper in JavaScript and get examples.
  • Watch the video commenting these codes
  • Study the exercises and test them with the grading infrastructure CodeGradX.

You may also change the way you learn every other day!

Links

The Web is full of documents related to JavaScript. Alas not all of them are worth reading. Below you may find some links I regularly use.

Exercises

Below is a link to an exercise served by the CodeGradX infrastructure. The CodeGradX infrastructure proposes exercises that are mechanically graded. You will be presented the stem of an exercise, then you write (or paste) your code in a JavaScript editor, send it to the mechanical grader and get back (roughly 15 seconds after) a grading report (containing a numeric mark if you are interested).

The CodeGradX infrastructure is currently run by a constellation of servers hosted elsewhere. In order to be able to return your achievements to EdX and to memorize your answers in your history, CodeGradX requires that you communicate your name and email. Your name will become your pseudo on CodeGradX, your email can be used if you forget your password and want to access directly CodeGradX.

So before trying the exercise, register with CodeGradX with the following button. After reading and accepting the User Agreement, you will have access to the whole set of programming exercises accompanying this MOOC.

The exercise proposed below is very simple: write a function that takes three numbers and return the lowest of them. Try it but don’t forget to register first!

⚙  min3.3

2  JavaScript - the language (Weeks 1 and 2)

2.1  Foreword

2.1.1  Foreword

Foreword

JavaScript is a rich language with numerous concepts. Unfortunately, most of the time, to show how to use one of these concepts requires to use other concepts so forward references cannot be avoided and quite often at the beginning, you will have to use concepts not entirely explained. This is life!

Moreover, JavaScript is a language that looks apparently very simple and similar to many other programming languages. However, this is wrong, JavaScript is full of oddities and definitely not trivial at all!

Therefore the first two weeks of that MOOC will present JavaScript, the language, topics after topics with two mantras:

  • Whole truth cannot be revealed immediately! But layer by layer.
  • Ultimate whole truth will not fit in a four-week course! Learning an evolving language is a full-time job.

During the first two weeks you are encouraged to peruse the documents, to hear the video commentaries and, concurrently, to code again and again; lots of exercises mechanically graded are available.

During the first week, I suggest that you study till the “Objects” section 🔍 but of course you are free to gofurther and study the rest of weeks 1&2.

Do not forget to also scan the appendices where you will find links towards useful documentation and information about testing, modules.

The third week is dedicated to concurrence-related features such as events, callbacks, promises, generators and workers.

The fourth and last week is dedicated to distribution that is harmony between browsers and servers including the use of WebSocket.

Eventually, you will have a very precise control of the JavaScript language in its ECMAscript 2015 incarnation, you will have first class knowledge of its most recent features and know how to use them to build multi-servers and multi-clients applications.

2.2  Presentation

2.2.1  JavaScript: a programming language

JavaScript: a programming language

  • Dynamically typed
  • Automatic memory management
  • Safe though lax
  • No classes but prototypes
  • Good compilation was not an initial goal
  • Created in 1995 by Brendan Eich for Netscape,
  • variants in every browser
  • now also usable for servers (Node.js)
  • Major overhaul with ECMAscript 2015 (ES6)
  • Funnier than ever!

2.3  Values

2.3.1  Taxonomy of values

Taxonomy of values

  • primitive: boolean, number, string, function, object
  • special, singleton: null, undefined, NaN
  • with syntactic support: array, regular expression
  • Predefined classes: Error, Date, Boolean, String, Object, etc.

2.3.2  Boolean

Boolean      🗁

  • Two values: true and false
  • Keywords for these two values: true, false
  • However some values are considered falsy, they are: false, 0, "" (the empty string), undefined, null and NaN
  • All values not falsy are considered truthy
  • typeof true is the string 'boolean'
  • Possible operators (à la C): &&, ||, !
  • Shortcircuit semantics for &&, ||

2.3.3  Numbers

Numbers

  • Internally only double precision (64 bits) floating point numbers (aka floats)
  • Hence integers on 54 bits [−253   …   2+53−1]
  • Special values: NaN, Infinity. both have type number
  • Some syntactic prefixes for integers 0x, 0o, 0b for non decimal bases
  • Possible operators (à la C) on numbers: +, -, *, /, %, ==, <, <=, etc.

Computing with numbers

  • All computations operate on floats
  • NaN (not a number) contagion
  • Infinity may be positive or negative
  • NaN is not equal to NaN

typeof 1 === "number"
typeof 1.2 === "number"
typeof 1/0 === "number"
typeof Math.sqrt(-1) === "number"
3 === 6/2
1.5 === 3/2
1/0 === 2/0

Useful utilities around numbers      👁       🗎       🗏

  // number -> string
1.0.toString()           === '1'
(1).toString()           === '1'
  // string -> number
parseInt("12.3")         === 12
parseFloat("12.3e-2")    === 0.123
  // number -> integer  
Math.round(1.53)         === 2
Math.floor(1.53)         === 1
  // recognizers:
typeof 2.3               === 'number'
Number.isInteger(2.3)    === false
isFinite(4/0)            === false
isNaN("foo"/2)           === true

2.3.4  Strings

Strings

  • Strings are immutable sequences of Unicode characters
  • Syntactic support enclosed with single or double quotes
  • and backslash (à la C). Special meaning for some backslashed characters such as \n, \', \", etc.
  • Insert Unicode characters with \uXX or \uXXXX
  • Syntactically restrained to a single (logical) line (except for backticked (or backquoted) characters)
  • No character only strings of length 1

Computing with strings

  • Concatenation with +
  • comparison with <, <=, >, >= and == according to Unicode ordering (see also the localeCompare() function)
  • String contagion: result is a string as soon as one argument is a string (or can be converted into a string)
  • Length (in characters) with the .length property
  • Characters counted (à la C) from zero
  • Strings can also be seen as arrays of characters
  • (interpolated) creation with backquotes

Strings examples      👁       🗎       🗏

"foo" + 42               === "foo42"
"\u03c0 is PI".length    === 7
"sin\
gle".length              === 6
`1+2=${1+2}`             === "1+2=3"
"foobar".substring(3, 5) === "ba"
typeof "foobar"          === "string"
"foobar".indexOf('b')    === 3
"foobar"[3] === 'b'      === true
console.log("a\nbc\n");   // prints 
a
bc

2.3.5  Functions

Functions

  • Defined with function
  • May have a name or be anonymous
  • Functions are first-class citizens (given as argument, returned as result or stored)
  • Result defined explicitly with return
  • Functions introduce a scope 🔍
  • And remember,
    • values have types
    • but variables don’t have types!

Function examples      🗁       👁       🗎       🗏

let surround = function (word, words) {
  return word + words + word;
};
function enclose (f, word, g) {
  return f(word) + word + g(word);
}
typeof enclose             === 'function'

(function fact (n) {
  if ( 0 < n ) {
    return n * fact(n - 1);
  } else {
    return 1;
  }
})(10)                     === 3628800

Function invocation      🗎       🗏

  • Arity is not checked at invocation
  • Missing arguments become undefined
  • Superfluous arguments are ignored
  • However all arguments (even superfluous) can be obtained through the pseudo-array arguments

function sum () {
  let result = 0;
  for ( let i=0 ; i<arguments.length ; i++ ) {
    result += arguments[i];
  }
  return result;
}
sum(sum(), sum(1, 2), 3);   // 6
// parameters are sum(), sum(1,2) and 3
// arguments are 0, 3 and 3
// Other version of sum see seq-array-methods

⚙  min3.3 ⚙  min4.2 ⚙  circlefn.1 ⚙  strcut.1

arguments and arrays      🗁       👁       🗎       🗏

  • arguments is not an array though it offers the length property.
  • A real array might be directly obtained with rest parameters (see uses of ... below)

function f1 () {
  return Array.from(arguments);
}
function f2 () {
  return [ ...arguments ];
}
function g (first, ...rest) {
  return rest;
}
// every next expression yields [1, 2, 3]
f1(1, 2, 3)    f2(1, 2, 3)  g(0, 1, 2, 3)

Function creation      👁       🗎       🗏

  • Functions can also be synthetized from text. This is inefficient and should be rare.
  • The returned function is not a closure and cannot use the local environment.

let factor = 5;
let f1 = (function (factor) {
  return new Function(['x', 'y'], 'return factor*x*y');
})(10);
f1(2, 3);   // 30 = 5*2*3

let f2 = (function (factor) {
  return new Function(['x', 'y'], 'return ' + factor + '*x*y');
})(10);
f2(2, 3);   // 60 = 10*2*3

let f22 = (function (factor) {
  return (new Function(['factor'],
    'return function (x, y) { return factor*x*y }'))(factor);
})(10);
f2(2, 3);   // 60 = 10*2*3

let f3 = (function (factor) {
  return function (x, y) { return factor*x*y };
})(10);
f3(2, 3);   // 60 = 10*2*3

2.4  Syntax

2.4.1  Main syntactic features

Main syntactic features

  • Syntax a la C
  • Comments a la C /* */ or C++ //
  • Instructions may be explicitly separated with semi-colons or implicitly with end of lines.
  • Most people recommend semi-colons at the end of instructions but that does not help as in:

                       // bad writing:
return a(x, f(y))    
     + b(y, f(x));

                       // better writing
return a(x, f(y)) +
       b(y, f(x));

2.5  Objects

2.5.1  Characteristics of objects

Objects and hashtables

  • Objects (in JavaScript) are associative arrays also known as hashtables
  • Objects are not objects (as in Object-Oriented Programming languages)
  • JavaScript objects contain properties, properties have a name and a value
  • Syntactic support for object creation with { key: expression, ... }
  • Access to properties with dotted syntax
  • Caution with final comma
  • As usual for hashtables, key-value pairs can be added, deleted, checked for existence (with keyword in) or enumerated 🔍

Objects examples      👁       🗎       🗏

let point = { x: 11, y: 22 };
typeof point                  === 'object'
point.x                       === point.y / 2;  
point.z = 33;
'z' in point                  === true;
delete point.x;
'x' in point                  === false
point.x                       === undefined
typeof point.x                === 'undefined'

Shortcut syntax      🗎       🗏

  • A shortcut syntax exists to create an object with properties initialized by variables when the properties and the variables have the same name.
  • Another shortcut syntax exists to set a property which name is computed.

let a = 1, b = 'c';
let o = { a, [b]: 2 };
// is the same as
let o = { a: a, c: 2 };

2.5.2  Properties

Properties

  • Properties names are strings (or anything that can be converted into a string)
  • Access to a property with a computed name is also available with square brackets (but less efficient!)

let me = { firstname: "Christian", nickname: "QNC" };
me['first' + 'name'] === "Christian"
('last' + 'name') in me === false

Enumerations

  • Properties may be enumerated with a loop such as for (let variable in object) ...
  • The array of property names can also be obtained with Object.keys()
  • Order of keys is implementation dependent

let me = { firstname: "Christian", nickname: "QNC" };
Object.keys(me)       === ['nickname', 'firstname']
for ( let key in me ) {
  me[key] += '.';
}
me.nickname           === 'QNC.'

🔍

2.6  Special values

2.6.1  Special values

null and undefined

  • undefined is returned by JavaScript when fetching a missing thing such as an unexistent property or an uninitialized variable.
  • undefined has type 'undefined'
  • You may use null in places that require an object and you don’t want to provide one or you want to provide an orphan object without any inherited property. See call or apply.
  • null has type 'object'
  • NaN is returned where a number is expected and the intended computation could not yield a number
  • NaN has type 'number'
  • NaN is not equal to NaN

2.7  Arrays

2.7.1  Characteristics of arrays

Characteristics of arrays      🗁

  • Arrays are ordered sequence of values
  • They may contain heterogeneous values
  • Created with new Array(initialLength) or square brackets
  • Accessed with a square brackets syntax
  • Arrays are objects with positive integer property names (starting with 0)
  • Arrays possess a .length property (also writable)
  • Dynamic structure with lots of methods mimicking stacks or queues

Arrays examples      👁       🗎       🗏

let t = [ 'a', 'b', [5+6, 'b'] ];
typeof t            === 'object'
Array.isArray(t)    === true
t[1]                === t[2][1]
t.length            === 3
t[9] = 'd';
t.length            === 10
t.length = 2;
t[2]                === undefined
t['c'] = 33;
t['c']              === t.c
t.length            === 2

⚙  hash2array.1 ⚙  histogram.1 ⚙  shuffle.2

Arrays length      🗎       🗏

  • The length of an array is the first available index greater than all existing indexes.
  • Modifying the length may reduce the number of terms in the array.
  • Maximal length is 232−1

Arrays keys      🗎       🗏

  • Arrays are objects hence indexes are keys are strings
  • Numeric keys are converted into strings

let t = [ 'z', 'a', 'b' ];
t[1]                         === 'a'
t[1.0]                       === 'a'
t[1 + 1e-16]                 === 'a'
t[1 + 1e-15]                 === undefined
t['1']                       === 'a'
t['1.0']                     === undefined

⚙  purify.1 ⚙  invertobject.1

Array methods      🗎       🗏

  • Stacks and queues with push, pop, shift, unshift
  • Browsing with forEach, entries, map, reduce
  • Search with index
  • Extraction/modification with splice and slice
  • and many others...

function sum () {
  return [...arguments].reduce((a, b) => a+b, 0);
}
sum(sum(), sum(1, 2), 3);   // 6

2.8  Regular expressions

2.8.1  Regular expressions aka RegExps

Regular expressions aka RegExps

  • Regular Expressions are syntactically supported, enclosed with /
  • They may also be created with new RegExp()
  • Regular expressions adopt the syntax of Perl regexps
  • Regexps are regular values: first-class citizens
  • Two main binary methods:
    • search on strings
    • exec on regexps

typeof /^fo+/ === 'object'
if ( "foobar".search(/^fo+/) >= 0 ) {
  let r2 = new RegExp("f(o+)(.*)a");
  return r2.exec("foobar", "g");
  // [ 'fooba', 'oo', 'b', index: 0, input: 'foobar' ]
}

2.9  Classes

2.9.1  Predefined classes

Predefined classes

  • Types are not classes!
  • Numerous predefined classes exist such as Date, Array, RegExp, Symbol, Error, TypeError, etc.
  • Besides types boolean, string, number exist predefined wrapper classes such as Boolean, String, Number
  • There are some automatic conversions between typed values and wrapped objects but they can be distinguished!
  • Predefined classes are often containers for utility functions and just used for them
  • The valueOf() method unwraps wrapped values.

Predefined classes specifics      🗁       👁       🗎       🗏

let s = "a string";
typeof s                  === 'string'
s instanceof String       === false
let S = new String(s);
typeof S                  === 'object'
S instanceof String       === true
s + S                     === "a stringa string"
S.length                  === s.length
typeof S.valueOf()        === 'string'

2.10  Comparisons

2.10.1  Equalities

Equalities

  • Identity is checked with ===. Identity means
    • same type,
    • same value for booleans or numbers (except NaN),
    • both null or both undefined
    • same content for strings
    • same reference for all other values (objects, arrays, functions)
  • Equality is checked with ==.
  • Identity implies equality
  • null and undefined are equal
  • If not identical, operands are converted to a common ground (strings or boolean towards numbers, objects to primitive values with toString() or valueOf()) and then compared again for equality.

Equalities examples      👁       🗎       🗏

All these expressions are truthy:

NaN !== NaN                  11 > 3
NaN != NaN                   "11" < "3"
3 !== 1/0                    "11" > 3
3 != 1/0                     1 < 1 + 1e-15
undefined === undefined      1 === 1 + 1e-16
null !== undefined           null == undefined
[1] != [1]                   {a: 1} != {a: 1}
"chaine" == "cha" + "ine"    "chaine" === "cha" + "ine"
11 == '11'                   [] == false

2.11  Coercions

2.11.1  Conversions

Conversions

  • JavaScript is rather lax
  • Many expressions do compute something

[4] * [4]       // 16
[] * []         // 0
[] * {}         // NaN
{} * []         // expected expression, got '*'
[4] * [4, 4]    // NaN
{} * {}         // NaN

2.12  Expressions

2.12.1  About expressions

About expressions

  • JavaScript expressions look like C or Java expressions. JavaScript adds to Java: in, typeof, delete
  • Expressions are constants (boolean, numbers, strings, null), variables, operators and their operands, function definitions and invocations, object creations (regexps, hashtable, arrays), etc.
  • precedence and associativity look normal.

Operators

  • on numbers + - * / , comparison,
  • on 32-bit integers (shift, and, or, not, xor)
  • on strings +, comparison
  • on boolean (and, or, not)
  • assignments = and variants += *= <<= and incrementation or decrementation ++ --
  • sequence ,
  • ternary alternative ?:

Instructions

  • As in C, expressions are instructions
  • Sequences of instructions (enclosed within curly braces) i.e., blocks
  • Alternative if with condition, consequence and optional alternant if else. Multiple cases are handled with switch (including default and break)
  • Loops while and do while including break and continue
  • Iterating loops with for, for in and for of
  • Confinement with try catch finally and throw
  • Within functions or generators, return and yield

2.13  Functions and variables

2.13.1  Variables and scope

Variables and scope      🗁

  • Variables can only be used in their scope.
  • The scope of a variables (resp. definition) is the area of text where that variable (resp. definition) is available.
  • The environment available in some place is the set of variables that can be used at that place.
  • Scope may be
    • global,
    • restricted to a function body or
    • restricted to a block.
  • Variables are introduced by var, by functions or by implicit references. With ECMAscript 2015, they may also be introduced with let and const.
  • Variables can exist or not and, when existing, be initialized or not.

Pattern matching arguments      🗁       👁       🗎       🗏

  • Instead of variables, patterns may be used

function f1 ({a = 1}) {
  return {a: a};
}
const f2 = ({a = 1}) => ({a});
f1({a: 3})    f2({a: 3})    // {a: 3}
f1({})        f2({})        // {a: 1}
f1()          f2()          // Error!

const g = ({a, b = 7}) => a+b;
g({a: 3})         // 10
g({b: 8, a: 3})   // 11

Scope examples      👁       🗎       🗏

let g0;    // global scope, g0 exists uninitialized
g1 = 1;    // global scope, g1 exists initialized
function f1 (l1) {       // l1 function scope
  const l2 = l1 + 1;     // l2 block scope
  if ( l1 ) {
    let l3 = 3;          // l3 block scope
    g2 = l3;             // g2 global scope
  }                      // l3 no longer visible
  let l5 = 2 * l2;       // l5 rest of block scope
  for ( let l4 in g0 ) { // l4 block scope
    l1 = l4;
  }                      // l4 no longer visible
}                        // l1, l2, l5 no longer visible
                         // f1 global scope

⚙  compose.1 ⚙  swapfn.1 ⚙  once.2

The old var      👁       🗎       🗏

var g0;   // global scope, g0 exists uninitialized
function f1 (l1) {       // l1 function scope
                         // l2, l3, l4 also visible here
  var l2 = l1 + 1;       // l2 function scope
  if ( l1 ) {
    var l3 = 3;          // l3 function scope
    g2 = l3;             // g2 global scope
  }
  for ( var l4 in l3 ) { // l4 function scope
    l1 = l4;
  }
}
                         // f1 global scope

Unexpected captures with var      🗎       🗏

let fnsA = [];
for ( var i=0 ; i<5 ; i++ ) {
   fnsA.push(function () { return i });
}
fnsA[0]() === fnsA[1](); 
fnsA[2]() === 5;
      let fnsB = [];          // First fix
      for ( var i=0 ; i<5 ; i++ ) {
        (function (ii) {
          fnsB.push(function () { return ii });
         })(i);
      }
      fnsB[2]() === 2
      fnsB[3]() === 3
let fnsC = [];          // Better fix!
for ( let i=0 ; i<5 ; i++ ) {
   fnsC.push(function () { return i });
}
fnsC[2]() === 2

Scopes visualization      🗎       🗏

  • v1 (resp. v2) do have the same scope: the body of the function (resp. the body of the closest enclosing function).
  • The scope of v3 is the rest of the enclosing block (after the assignment sign). The scope of v4 is the body of the for loop after the in word.
  • As soon as a variable is available, it can be enclosed in a closure. 🔍.

2.14  Types

2.14.1  The typeof operator

The typeof operator

  • The typeof operator takes an expression as operand, evaluates it and returns its type that is 'number', 'string', 'boolean', 'object', 'function' or 'undefined'
  • If the operand is an unexistent variable, this is not an error, its type is 'undefined'.

(function () {
    // veryWeirdName is not global:
    global.veryWeirdName === undefined;
    try {
      return veryWeirdName;
    } catch () {
      // ReferenceError: veryWeirdName is not defined
      typeof veryWeirdName === 'undefined';
    }
})();

2.15  Functions

2.15.1  The function declaration

The function declaration

  • The function keyword creates a function
  • As a non anonymous declaration, function assigns the created function to a variable
  • As a non anonymous expression, function allows the name to be used in the function body.

Alternate syntax examples      🗁       👁       🗎       🗏

function fact1 (n) {
  return n>1 ? n*fact1(n-1) : 1;
}
let fact2 = function fact3 (n) {
  return n>1 ? n*fact3(n-1) : 1;
}
let fact4 = function fact5 (n) {
  return n>1 ? n*fact4(n-1) : 1;
}
let fact6 = (function () {  // similar to fact2
  function fact7 (n) {
    return n>1 ? n*fact7(n-1) : 1;
  }
  return fact7;
})();
fact1(4) === fact2(4)
fact1(4) === fact4(4)
fact1(4) === fact6(4)

let oldfact1 = fact1;
let oldfact2 = fact2;
let oldfact4 = fact4;
let zero = () => 0;
fact1 = fact2 = fact4 = zero;
oldfact1(4) === 24
oldfact2(4) === 24
oldfact4(4) === 0

Arrow notation      🗎       🗏

  • Functions can also be declared with =>
  • Parentheses around a single variable may be omitted
  • curly braces around a body containing a single expression may be omitted as well as the return keyword

let fact1 = (n) => n>1 ? n*fact1(n-1) : 1;
let fact2 =  n  => n>1 ? n*fact2(n-1) : 1;
let fact3 =  n  => {
  return n>1 ? n*fact3(n-1) : 1;
};

🔍

2.15.2  Embedded scopes

Embedded scopes      🗁       👁       🗎       🗏

  • Functions can be defined within functions.

function subtract (x, y) {
  function opposite (x) { return -x; }
  function sub (x, y) { return x - y; }
  if ( y ) {
    return sub(x, y);
  } else {
    return opposite(x);
  }
}
function subtract2 (x, y) {
  function opposite (x) { return -x; }
  function sub (x, y) { return x - y; }
  return (y ? sub : opposite)(x, y);
}

⚙  toggler.1 ⚙  memoize.1 ⚙  curry.1

Order of evaluation      🗎       🗏

  • In the body of a function, all function declarations are processed first (to ensure mutual recursion)

function f2 () {
  function f21 (x) { return 2 * x; }
  return f22(3);
  function f22 (n) { return f21(1 + n); }
}
f2();    // 8

2.16  Global environment

2.16.1  Characteristics of global environments

Characteristics of global environments

  • The value of a variable is looked up first in the local environment from the nearest enclosing function to the most outer enclosing function
  • If not found, it is looked up in the global environment.
  • There is always a global environment.
  • The global environment is represented by an object (a hashtable).
  • In a browser window, this is the value of window
  • In Node.js, this is the value of global

let gg = 44;
global.gg === global['g' + 'g'];
var ggg;
global.ggg === global['ggg'];
ggg = 55;
global.ggg === global['ggg'];

2.17  Closures

2.17.1  Characteristics of closures

Characteristics of closures      🗁       👁       🗎       🗏

  • All functions are closures.
  • A closure is some code paired with the environment where that closure was defined.
  • The value of a variable is looked up first in the environment of the enclosing function then in the environment where that closure was defined, etc.

function makePrefixer (prefix) {
  return function (word) {
    return prefix + ' ' + word;
  };
}
let helloer  = makePrefixer('hello');
let welcomer = makePrefixer('welcome');
helloer('world');    // hello world
welcomer('home');    // welcome home

Closures and free variable      🗎       🗏

  • Alternate but similar definition
  • prefix is free in prefixer
  • prefix is said to be closed in (captured by) prefixer

function makePrefixer2 (prefix) {
  function prefixer (word) {
    return prefix + ' ' + word;
  }
  return prefixer;
}

Closures and mutable variables      🗁       👁       🗎       🗏

  • When a function is invoked, its variables are bound to the arguments.
  • The invocation creates new bindings.
  • Closures capture bindings not values.

function makeBox (content) {
  return { get: function () {
                 return content;
           },
           set: function (v) {
                 content = v;
           }
  };
}

const box = makeBox(10);
box.get() == 10;
box.set(22);
box.get() == 22;

Poor man’s module      🗁       👁       🗎       🗏

let fastSine = (function () {
    
  let table = [];
  for (let i=0 ; i<1000; i++) {
    table[i] = Math.sin(2*Math.PI*i/1000);
  }
  return function (x) {
    let index = Math.abs(x)%(2*Math.PI);
    index = index/(2*Math.PI);
    let value = table[Math.round(1000*index)];
    return (x>0)?value:(-value);
  };
  
})();

fastSine(Math.PI/5) // close to Math.sin(Math.PI/5)

2.18  Objects

2.18.1  Generation of objects

Generation of objects

  • JavaScript is not an Object-Oriented Language but a Prototype Based Language.
  • Objects (in OOP-sense) are created by classes acting as waffle makers: all waffles do have the same behaviors (methods).
  • Object (in Prototype sense) are created by cloning, the daughter cell inherits all the behaviors the mother had at cloning-time. After cloning both may evolve independently.

Behaviors      👁       🗎       🗏

  • Methods (in OOP parlance) are functions stored in properties of an hashtable (an object in JavaScript parlance)

let o = {
  a: 2,
  m1: function (b) {
    return this.a + b;
  }
};
o.m1(5);
o.m2 = function (c) {
  return this.a * c;
};
o.m2(5);

Invocation and this      🗎       🗏

  • Two ways to invoke a function

  f(2, x+3)
o.f(2, x+3)

  • When a function is invoked on an object as a method, the keyword this is bound to that object
  • When a function is not invoked as a method, the keyword this is also bound to something.

The this keyword      👁       🗎       🗏

  • this is a keyword not a variable
  • this cannot be closed over by function
  • Invoking a function binds a new this

function Foo () {               
                       // let self = this;
  this.count = 0;               
  this.mkincr = function (d) {  
    return function () {        
      return (this.count += d);
                       // return (self.count += d);
    }                             
  }                               
}                               
let vf = new Foo();              let vf = new Foo();
let f = vf.mkincr(2);            let f = vf.mkincr(2);
f(); /* missing this! */         f();
vf.count == 0;                   vf.count == 2;
f.call(vf);     // Fix
vf.count == 2;

this and =>      👁       🗎       🗏

  • this is not changed by =>
  • or can be explicitly specified in invocations

let a = [ 10, 20, 30 ];
function squareArray (t) {
  t.forEach(function (item, i) {
    this[i] = item * item;          // Wrong!
  });
  return t;
}
squareArray(a); // [10, 20, 30]

function squareArray (t) {
  t.forEach(function (item, i) {
    this[i] = item * item;
  }, t);                            // Fix!
  return t;
}
squareArray(a); // [100, 400, 900]

// Turn squareArray into a method:
function squareArray () {    // with =>
  this.forEach((item, i) => {
    this[i] = item * item;
  });
  return this;
}
Object.getPrototypeOf(a).squareArray = squareArray;
a.squareArray(); // [100, 400, 900]

function squareArray () {    // with =>
  const square = (item, i) => {
    this[i] = item * item;
  };
  this.forEach(square);
  return this;
}
Object.getPrototypeOf(a).squareArray = squareArray;
a.squareArray(); // [100, 400, 900]

Invocation      🗁       👁       🗎       🗏

  • Functions are also objects with some specific invocation methods: call, apply and bind

function countargs () {
  return Object.keys(this).length + arguments.length;
}
countargs.call({}, 'a');              // 1
countargs.call({}, 'a', 'b');         // 2
countargs.call({x:3}, 'a', 'b');      // 3
countargs.apply({x:3}, ['a', 'b']);   // 3
let f = countargs.bind({x:3});
f('a', 'b');                          // 3

function trace (f) {
  return function (...args) {
    console.log("calling with", args);
    let result = f.apply(this, args);
    console.log("result is", result);
    return result;
  };
}
let cat = trace(countargs);
cat.call({x:3}, 'a', 'b');
// calling with [ 'a', 'b' ]
// result is 3

2.18.2  Functions and hashtables

Functions and hashtables      🗁       👁       🗎       🗏

  • Functions are also hashtables

function incr (x, dx) {
  dx = dx || incr.default.dx;
  return x + dx;
}
incr.default = { dx: 1 };
incr(20, 2);               // 22
incr(20);                  // 21

Functions and default arguments      🗁       🗎       🗏

  • Default value may be specified for variables

function incr (x, dx = 1) {
  return x + dx;
}
incr(20, 2);               // 22
incr(20);                  // 21

const decr = (x=100, dx) => x-dx;
decr(105, 3)               // 102
decr(undefined, 3)         //  97

⚙  checkargs.1

2.19  Prototypes

2.19.1  The new keyword

The new keyword      👁       🗎       🗏

  • new allocates an object,
  • adds some internal information (constructor and prototype),
  • then calls a user’s function to initialize the object.

function Thing (stuff) {
  this.myKind = 'Thing';
  this.stuff = stuff;
  // instance behavior
  this.setStuff = function (stuff) {
    this.stuff = stuff;
  }
}
let t1 = new Thing(1);
let t2 = new Thing(2);

t1.stuff === 1
t2.setStuff(3); t2.stuff === 3

t1.setStuff != t2.setStuff;

t1.binz = function () { return 'binz' };
t1.binz(); // yields 'binz'
t2.binz(); // TypeError: undefined is not a function

Constructor and instanceof      🗎       🗏

  • An object is linked to its constructor when created with the new keyword
  • Constructors are functions
  • All functions are associated to a prototype.
  • instanceof links objects and their constructor

function Thing (stuff) {
  this.myKind = 'Thing';
  this.stuff = stuff;
}
let t1 = new Thing(1);
let t2 = new Thing(2);
t1.constructor === Thing;
t1 instanceof Thing === true

Constructor visualization      🗁       🗎       🗏

  • Print this figure and pin it to your wall. This is the most important thing to memorize from JavaScript!
  • Green is for implementation pointer, pink for the special constructor property, black for regular properties.

Constructor property      🗁       👁       🗎       🗏

  • The constructor property is writable
  • but not used to compute instanceof
  • You should have good reasons if you want to change it!

function Thing (stuff) {
  this.myKind = 'Thing';
  this.stuff = stuff;
}
let t1 = new Thing(1);
t1 instanceof Thing === true

t1.constructor = undefined;
t1 instanceof Thing === true

delete t1.constructor;
t1 instanceof Thing === true

const oldThing = Thing;
function Thing () {}
t1 instanceof Thing === false
t1 instanceof oldThing === true

return and Constructor      👁       🗎       🗏

  • By default, a constructor terminates with an implicit return this
  • However, you may return another value

function Singleton (initial) {
  if ( ! Singleton.default ) {
    Singleton.default = initial;
  }
  return Singleton.default;
}
let o1 = new Singleton({a: 1});
let o2 = new Singleton({b: 3});

o2.a === 1
o1 === o2

⚙  spycount.1 ⚙  memoize.1

2.19.2  Property lookup

Property lookup      👁

  • All objects have a prototype
  • The prototype of an object is obtained with Object.getPrototypeOf()
  • To access a property such as o.p, p is first looked up in o, then in the prototype of o, then in the prototype of the prototype of o, etc.

⚙  protochain.1 ⚙  getcolor.1

Property lookup visualization

Sharing behaviors

  • Store properties common to all instances of a constructor in the prototype of that constructor.

function Thing (stuff) {
  this.myKind = 'Thing';
  this.stuff = stuff;
}

Object.getPrototypeOf(Thing) !== Thing.prototype

let t1 = new Thing(1);
t1.other = 111;
let t2 = new Thing(2);

Thing.prototype.binz = function (d) {
  return (this.stuff += d);
}
t1.binz(10);
t2.binz(10);

Sharing behaviors visualization

Prototype handling

  • Object.create(p) creates a new object which prototype is p
  • Object.setPrototypeOf(op) sets the prototype of o to be p
  • Sharing properties does not depend on constructors

let o = {};
let p = Object.create(o); // cloning
Object.getPrototypeOf(p) === o;
Object.getPrototypeOf(Object) === null;

⚙  clone.1

Property mutation      👁

  • Mutating a property is done in the object itself
  • A sort of copy-on-write

let p = { a: 1 };
let o = Object.create(p);
p.a === o.a;
Object.keys(o); // []
o.b = 2;
Object.keys(o); // ['b']
p.b === undefined
o.a = 3;
Object.keys(o); // ['a', 'b']
p.a < o.a;
p.c = 4;
Object.keys(o); // ['a', 'b']

Property owned or inherited

  • Properties can be inherited from the prototype
  • or be proper to the object
  • Object.keys(o) returns the names of the proper (enumerable) properties of o
  • while for (let key in o) ... enumerates all the (enumerable) proper and inherited properties of o
  • Object.getOwnPropertyNames(o) returns the names of all proper (enumerable or not) properties of o
  • while o.hasOwnProperty(prop) checks whether prop is a name of a proper property of o
  • Finally prop in o checks whether prop is a property (proper or inherited, enumerable or not) of o

Property characteristics      🗁

  • Properties have several characteristics
    • a name and a value
    • read-only or writable
    • enumerable or not. Non enumerable means that the property does not appear in the result of Object.keys() nor in the result of for (let key in o) ...
    • configurable or not. Configurable means that the characteristics of the property may be changed and the property may be deleted.
  • Properties may be defined or fine-tuned with Object.defineProperty(objpropdescriptor)
  • Property description may be obtained via Object.getOwnPropertyDescriptor(objprop)

Property characteristics

let ob = { a: 1 };
Object.defineProperty(ob, 'bb', {
  enumerable:   false,  // hidden
  configurable: false,  // cannot be deleted or modified
  writable:     false,  // immutable hence constant
  value:   813
});
ob.bb;                   // 813;
Object.keys(ob);         // ['a']
ob.hasOwnProperty('bb'); // true
ob.bb += 1; ob.bb;       // 813
delete ob.bb; ob.bb;     // 813

Property accessors      🗁

  • Functional getter and setter may also be defined with Object.defineProperty(objpropdescriptor)
  • These getter and setter are used with a property syntax not an invocation syntax.

let clock = { current: 0 };
Object.defineProperty(clock, 'counter', {
  get: function() { return ++this.current; },
  set: function(newValue) {
    return (this.current = newValue);
  }
});
clock.counter;       // 1
clock.counter;       // 2
clock.counter = 9;   // 9
clock.counter;       // 10
Object.keys(clock);  // ['current']

Object extensibility

  • Objects are, by default, extensible: they may acquire new (own) properties
  • After Object.preventExtensions(obj), new (own) properties cannot be added to obj.
  • Object.isExtensible(obj) recognizes such objects.
  • If an object is not extensible and all its own properties are non configurable (cannot be removed) then the object is sealed and recognized by Object.isSealed(obj). Own properties may still be writable!
  • Object.freeze(obj) marks an object as immutable. Such objects may be recognized with Object.isFrozen(obj) A frozen object is not extensible, its own properties are read-only and its own properties descriptions cannot be modified. Inherited properties are not impacted!

2.19.3  Iterator and iterable

Iterator protocol      🗁       👁       🗎       🗏

  • An iterator is an object with a next method that returns a value with two properties: value and a boolean done
  • done is true when the iterator is exhausted.

let o = (function (things) {
  return {
    next: function () {
      if ( things.length > 0 ) {
        return { value: things.shift(),
                 done: false };
      } else {
        return { done: true };
      }
  }};
})(['a', 'b']);
let r = o.next();
r.value   // 'a'
r.done    // false
r = o.next();
r.value   // 'b'
r = o.next();
r.done    // true

Iterable interface      🗁       🗎       🗏

  • An iterable object has a specific method that returns an iterator
  • Iterable objects can be iterated with for of
  • The name of the specific method is the value of the global variable Symbol.iterator

let o = (function (things) {
  return {
    next: () => {
      if ( things.length > 0 ) {
        return { value: things.shift(),
                 done: false };
      } else {
        return { done: true };
      }
  }};
})(['a', 'b']);
let iterable = { [Symbol.iterator]: () => o };
for ( let v of iterable ) {
  console.log(v); // prints 'a', 'b'
}

2.20  Inheritance

2.20.1  Class inheritance

Class inheritance      🗁       👁       🗎       🗏

  • Sub-classing and calls to super

function Worker (name) {
  this.name = name;
};
Worker.prototype.getSalary = () => 1;

function Plumber (nickname) {
  Worker.call(this, nickname);  // super()
  this.nickname = nickname;
};
Object.setPrototypeOf(Plumber.prototype,
                      Worker.prototype);
Plumber.prototype.getSalary = function () {
              // super.getSalary
  return 10 * Worker.prototype.getSalary();
};

let joe = new Plumber('joe');
joe.nickname === joe.name;
joe.getSalary() === 10;
joe instanceof Plumber;  // true
joe instanceof Worker;   // true

⚙  pointcolore.2 ⚙  add-method.1 ⚙  cascade.1 ⚙  doesnotunderstand.1

Class inheritance visualization      🗎       🗏

Class inheritance revisited      🗁       🗎       🗏

  • Variant using the constructor property (supposed immutable)

function Worker (name) {
  this.name = name;
};
function Plumber (nickname) {
  const ogt = Object.getPrototypeOf(this);
  Object.getPrototypeOf(ogt).constructor
    .call(this, nickname);
  this.nickname = nickname;
};
Plumber.prototype = Object.create(Worker.prototype);
Plumber.prototype.getSalary = function () {
  const ogt = Object.getPrototypeOf(this);
  return 10 * Object.getPrototypeOf(ogt)
    .getSalary.call(this); 
};

Class inheritance in ECMAscript 2015      👁       🗎       🗏

class Worker {
  constructor (name) {
    this.name = name;
  }
  getSalary () {
    return 1;
    }
}
class Plumber extends Worker {
  constructor (nickname) {
    super(nickname);
    this.nickname = nickname;
  }
  getSalary () {
    return 10 * super.getSalary();
  }
}

Class expression      🗎       🗏

  • Classes may be dynamically computed
  • The class to extend can also be computed
  • However, this can only be used after calling super

function LastInstanceMemorizer (klass) {
  return class extends klass {
    constructor (...args) {
      super(...args);
      return klass.lastInstance = this;
    }
  };
}
const LIM_Plumber = LastInstanceMemorizer(Plumber);
let plumber1 = new LIM_Plumber('Mario');
plumber1 === LIM_Plumber.lastInstance
let plumber2 = new LIM_Plumber('jill');
plumber2 === LIM_Plumber.lastInstance

⚙  singletoner.1

2.21  Exercises of style

2.21.1  Interlude

Interlude: Counter      👁       🗎       🗏

  • JavaScript proposes more than one way to program
  • Program a counter with two operations increment and reset

Functional counter      🗁       🗎       🗏

function makeFunctionCounter () {
    let counter = 0;
    function increment () {
        return ++counter;
    }
    function reset (value) {
        return (counter = value);
    }
    return function () {
        if ( arguments.length > 0 ) {
            return reset(arguments[0]);
        } else {
            return increment();
        }
    };
}
let c1 = makeFunctionCounter();
c1()  === 1      // increment
c1()  === 2      // increment
c1(5) === 5      // reset
c1()  === 6      // increment
typeof c1 === 'function'

The counter is private and can only be accessed via the interface. However there is no way to know if such a function is a counter. The counter is accessed via a get/set technique but all counters have their own closures reset and increment.

Class-less object counter      🗎       🗏

function makeObjectCounter () {
    return {
        counter: 0,
        increment: function () {
            return ++this.counter;
        },
        reset: function (value) {
            return (this.counter = value);
        }
    };
}
let c1 = makeObjectCounter();
c1.increment() === 1
c1.increment() === 2
c1.reset(5)    === 5
c1.increment() === 6
typeof c1 === 'object'
c1.counter // not encapsulated

The counter is not private, it can be modified by anyone. The counter is a raw object without more specific type however the interface shows explicitly that the object supports increment and reset methods. All counters have their own closures reset and increment.

Class-less object counter with accessors      🗁       🗎       🗏

function makeObjectWithAccessors () {
    let counter = 0;
    return {
        set value (value) {
            return (counter = value);
        },
        get value () {
            return ++counter;
        }
    };
}
let c1 = makeObjectWithAccessors();
c1.value === 1
c1.value === 2
c1.value = 5
c1.value === 6
typeof c1 === 'object'

The counter is private and can only be accessed via the interface. However there is no way to know if such a function is a counter. The counter is accessed as a regular property. All counters have their own closures reset and increment.

Classy object counter      🗁       🗎       🗏

class Counter {
  constructor () {
    this.counter = 0;
  }
  reset (value) {
    return (this.counter = value);
  }
  increment () {
    return ++this.counter;
  }
}
let c1 = new Counter();
c1.increment() === 1
c1.increment() === 2
c1.reset(5) === 5
c1.increment() === 6
typeof c1 === 'object'
c1 instanceof Counter === true
c1.counter // not encapsulated

The counter is not private, it can be modified by anyone. The counter is an object of class Counter to which new methods can be added. However the interface shows explicitly that the instance supports increment and reset methods. All counters share the code of their methods.

Classy object with private counter      🗁       🗎       🗏

class PrivateCounter {
  constructor () {
    let counter = 0;
    this.increment = function () {
        return ++counter;
    }
    this.reset = function (value) {
        return (counter = value);
    }
  }
}
let c1 = new PrivateCounter();
c1.increment()  === 1      // increment
c1.increment()  === 2      // increment
c1.reset(5) === 5          // reset
c1.increment()  === 6      // increment
typeof c1 === 'object'
c1 instanceof PrivateCounter
c1.counter === undefined

The counter is private and can only be accessed via the interface. The counter is an object of class PrivateCounter to which new methods can be added. However the interface shows explicitly that the instance supports increment and reset methods. All instance have their own closures reset and increment.

2.22  Library

2.22.1  Library

Library

  • A language is defined by its syntax, its operators, keywords and their meaning
  • A language also rests on predefined libraries (functions, classes, methods, etc.)
  • and is enriched by more and more new libraries (for instance, DOM within browsers, the process object for Node.js) but does not depend on them.
  • JavaScript library defines variables, functions, classes and methods.

Library classes and methods

  • More or less reflective functions in Object
  • Predefined methods on Strings (length, match, etc.), on Arrays (length, forEach, etc.), on Functions (call, apply, bind), on RegExps, etc.
  • The Math module with max, min, trigonometric lines, sqrt, logarithm, etc.

2.23  Conclusions on the language

2.23.1  Conclusions

Language main features

  • Taxonomy of values, see 🔍
  • Like other programming languages: operators, expressions, instructions
  • Values are typed, variables are not typed. See typeof and instanceof
  • Functions are closures; functions are first-class values. Arity is not checked, see arguments
  • Objects are hashtables holding properties. Properties may be characterized and enumerated. Nearly every value is an object (arrays, functions)
  • Methods are functions held in properties, see this
  • Objects are associated to a chain of prototypes to which they may delegate.
  • Classes, methods and super invocations are now available on top of prototypes.

Duck typing

  • The “type” of an object in JavaScript is defined as the set of properties and behaviors this object possesses.
  • If it quacks (i.e., it has a quack property, a nullary function) then this is a duck.

Conclusions

  • JavaScript
    • has numerous features,
    • has rules with exceptions,
    • has weird coercions hence its laxness
  • JavaScript offers a variety of programming styles:
    • functional,
    • prototype-based,
    • object-oriented
  • but JavaScript is fun!

Other interesting resources: JavaScript Garden

3  Concurrency (Week 3)

3.1  Foreword

3.1.1  Foreword

Foreword

Concurrency is a big word in the context of JavaScript since the code you write is run sequentially even if, underneath, the runtime looks like being concurrent. For instance, the runtime knows how to request two servers concurrently but, when these requests are completed, it will signal your code only sequentially.

The appearance of concurrency is given via numerous linguistic features, conventions and usages. In this section, are successively presented:

  • Orphan computations
  • Callback
  • Event
  • Promise
  • Generator
  • Worker

3.2  Presentation

3.2.1  Event loop

Event loop      🗁

  • JavaScript is single-threaded that is, it evaluates only one thing at every moment.
  • JavaScript is non preemptive that is, when some of your code runs, this code will run it until it terminates.
  • JavaScript is organized around an event loop:
    • This event loop maintains a list of tasks to run.
    • Whenever JavaScript finishes one task, it will pick and run another one from the list.
    • Whenever something external happens (an HTTP request comes back, a database query returns, a click on a button, etc.) the associated handler to process that event is inserted in the list of tasks to run.

Single thread

  • JavaScript is single-threaded
  • Therefore
    • no need for atomic section
    • no need for synchronized methods (in Java parlance)
  • When your code runs, no other code runs: you are alone
  • Therefore active wait blocks everything!

Non preemptive      🗁

  • JavaScript is non preemptive
  • When your code runs, you cannot be interrupted, you run till your code terminates.
  • Of course, you should terminate one day!

setTimeout(() => console.log("I run"), 0);
while ( true ) { } // never ends
// so "I run" will not be printed

Orphan computation      🗁       👁       🗎       🗏

  • With setTimeout, functions can be invoked after a given delay (at least)
  • these functions are only useful for their side-effects
  • the value they return will be ignored
  • the exception they might throw will be ignored (and thus becomes an uncaught exception)
  • the body of the function is considered as an orphan computation.

let timeout = setTimeout(
   function () {
     // perform something...
   }, 100);  // wait at least 100 milliseconds
clearTimeout(timeout);  // revocation

Uncaught exception      🗎       🗏

  • Uncaught exceptions are bad!
  • Uncaught exception are produced when orphan computation errs!
  • They may be noticed (with implementation-dependent features)
  • but the context of the exception is probably lost.

// Node
process.on("uncaughtException",
  function(err) {...});

// browsers
window.onerror =
  function(messageOrEvent, source, lineno, colno, error) {
    ...};

Computation chunks      🗎       🗏

  • Some operations have unbounded duration (file access, HTTP response, database query, waiting for a click, etc.)
  • Busy waiting blocks the process, the user interface, responses to events, etc.
  • So computation should be composed of reactions to events.

3.3  Callback

3.3.1  Callback

Callback

  • Operations which duration is unknown often takes an additional argument: a callback representing how to process the result of the operation.
  • Reading a file, querying a database, fetching an URL, parsing a document, waiting for a click are asynchronous.
  • An asynchronous operation returns normally as soon as the callback is registered.
  • The callback might be invoked later, if at all.
  • Invoking the callback is an orphan computation therefore the value returned by a callback or the thrown exception will be ignored.

Node callback convention

  • A callback in Node takes two arguments: an error and some data (in that order).
  • if the error is null then the data is meaningful.

const fs = require('fs');
fs.readFile("file/name", 'utf8',
  function (error, data) {
    if ( error ) {
      // handle the error
    } else {
      // do something with data
    }
  });

Callback common mistakes      👁       🗎       🗏

function test (filename, callback) {
  return fs.readFile(filename, callback); // BAD
}
try {
  const whatever = (e, c) => 'whatever';
  test(undefined, whatever); // throws TypeError
} catch (exc) {
  exc // TypeError: path must be a string or Buffer
}
try {
  test('/does/not/exis.ts', function (error, c) {
    // Error: ENOENT: no such file or directory
    throw error;  // throws nowhere!
  }) ===  undefined
} catch (exc) {
  // not evaluated!
}
let result = test('/file/that/exists',
     function cb_ok (e, content) {
       return content.length;   // returns nowhere!
     });
result === undefined

Callback hell      👁       🗎       🗏

  • How to combine three asynchronous tests in Jasmine
  • and check the compute function taking a callback ?

describe('combine tests', function () {
  it('combine 3 tests', function (done) {
    compute(..., function (error1, result1) {        // 1st test
      expect(error1).toBe(null);
      expect(result1).toBe(...);
      compute(..., function (error2, result2) {      // 2nd test
        expect(error2).toBe(null);
        expect(result2).toBe(...);
        compute(..., function (error3, result3) {    // 3rd test
          expect(error3).toBe(null);
          expect(result3).toBe(...);
          done();                                // end of tests
        }); }); }); });
});

Synchronous or asynchronous callback      🗎       🗏

  • With the next two functions, the callback is invoked with the Euclidian quotient and remainder of n and p.
  • Both prints 19 15
  • However the second always returns undefined and the result of the callback is lost.
  • The second callback is said to be asynchronous since it is run as an orphan computation.

function cbdivideSync (n, p, cb) {
  let q = Math.floor(n/p);
  let r = n - p * q;
  return cb(q, r);
}
function cbdivideAsync (n, p, cb) {
  let q = Math.floor(n/p);
  let r = n - p * q;
  setTimeout(() => cb(q, r), 0);
}

cbdivideSync(813, 42, console.log)
// prints 19 15
cbdivideAsync(813, 42, console.log)
// prints 19 15

⚙  cbgcd.1 ⚙  cbchain.1

3.4  Events

3.4.1  Events

Events

  • In Node, many objects emit various kinds of events; these objects are event emitters. Events are received by functions named listeners.
  • Within browsers, user’s interactions, AJAX requests, etc. also emit events received by listeners.

// In Node.js or within browsers (using JQuery):
emitter.on('eventName', function (...args) { ... });
emitter.emit('eventName', ...args);

Event emitters      🗁

  • The on (or addListener) method of an event emitter adds a listener to that emitter on a specific event name.
  • On a given emitter and an event name, more than one listener may be added.
  • Listeners can be removed with removeListener or removeAllListeners.
  • Listeners may be obtained with the listeners method.
  • The emit method of an emitter emits a named event accompanied with some data. All the related listeners are then invoked synchronously and orderly.
  • The values returned by listeners are ignored but a thrown value will stop the invocation of the remaining listeners and will become the value returned by emit.

Event emitters example      🗁       👁       🗎       🗏

// In Node.js
const EventEmitter = require('events');
class MyEmitter extends EventEmitter {}
const myEmitter = new MyEmitter();

{
  myEmitter.on('newListener', function (name, f) {
    myEmitter === this;
    console.log(`Got new listener on ${name}`);
  });
  myEmitter.on('error', console.log);
  // displays 'Got new listener on error'
  myEmitter.on('yell', function (sentence) {
    console.log(sentence.toLowerCase());
  });  // displays 'Got new listener on yell'
  myEmitter.on('yell', function (sentence) {
    console.log(sentence.toUpperCase());
  });  // displays 'Got new listener on yell'
  myEmitter.emit('yell', 'Hello world');
  // displays 'hello world'
  // displays 'HELLO WORLD'
}

Event emission      🗎       🗏

  • Listeners are invoked synchronously and orderly.
  • To emit an event is similar to the following code:

// myEmitter.emit(name, ...args)
myEmitter.listeners(name)
   .forEach((listener) =>
       listener.apply(this, [...args]);
);

Comparing event and callback      🗎       🗏

  • Listeners are invoked synchronously, callbacks can be synchronous or asynchronous (and mostly run as orphan computation).
  • An event may be multiply emitted while an asynchronous callback is often called only once.
  • For a single event, more than one listener can be present. Only one callback can appear where a callback is needed.
  • Listeners can be added or removed dynamically.
  • Adding, removing, processing listeners signal events.
  • Event listeners must know their event emitter, the reverse is not necessary.

⚙  listener.1 ⚙  completion.1

3.4.2  Event in browsers

Event in browsers

  • DOM event listeners in browsers are somehow different. See excellent figure from W3C documentation .
  • Listeners can be either capturing listeners or bubbling listeners See the useCapture argument of the DOM method target.addEventListener(typelistener[, useCapture]).
  • Capture phase When a click arrives, DOM elements from the biggest to the smallest (excluded) are enumerated and their capturing listeners are invoked. The stopPropagation method may abort this process.
  • Target phase The listeners of the smallest element (the target) are then invoked. The preventDefault method suppresses the default behavior. The stopPropagation method may abort this process.
  • Bubbling phase If the event is bubbles-able, the DOM elements from the smallest to the biggest are enumerated and the bubbling listeners are invoked. The stopPropagation method may abort this process.

Event summary

  • Many features of node are event emitters (http.get, etc.)
  • The error event is often used to signal an error for an event emitter.
  • Callbacks are used when a computation can yield a result or an error.
  • Events are used when a computation can signal many other things in supplement: progress, state change, etc.
  • However all events emitted without listeners are lost.

3.5  Promise

3.5.1  Promise

Promise      👁       🗎       🗏

  • A Promise is a placeholder for the result of some computation.
  • One may attach to a promise, functions to further its success (also called fulfillment) or its failure (also called rejection).
  • These sequels may be attached during or even after the computation of the promise.
  • Once the computation of a promise is terminated with a success or a failure, the promise is known as resolved (fulfilled or rejected) and the status of the promise cannot be altered.
  • cf. Promise specification
  • To resume, when a promise is created, its state is pending. Once terminated or resolved, its status may be fulfilled or rejected.

Promise basic signatures      🗎       🗏

  • A pair of functions to handle success and failure can be attached with the then method of a promise
  • A function to handle a failure can be attached with the catch method of a promise
  • Sequels are run as orphan computations.

let promise = ...
promise.then(success, failure)
promise.then(success)
promise.catch(failure)
promise.then(success).catch(failure)

// with success(result)
// and  failure(exception)

Promise visualization      🗎       🗏

  • Failures can occur within the promise computation but also within the sequels. An uncatched throw in a sequel induces a failure.
  • Failures can be caught by catch sequel and fixed. A failure is fixed when a catch sequel returns normally.

The next figure illustrates the differences between a binary then and a sequence of then-catch. Red stars represent the occurrence of a failure. Question marks represent an uncaught failure.


The first segment to the left represents the computation of the body of the promise which may return a value or throw an exception (marked with a red star). Segments under s or f represents the computation of the success or failure sequels.

The figure on the right shows that a failure in the success sequel can be fixed by the failure sequel. However in all cases, if the failure sequel fails, the failure will become uncaught.

Promise example of use      👁       🗎       🗏

  • Let us suppose that pdivide takes two numbers and returns a promise.
  • A success sequel of this promise will receive an object with properties quotient and remainder
  • But if the divider is zero, then a failure sequel wil receive a “division by zero” error.

let promise = pdivide(25, 4);
promise.then(console.log, function () => { throw 0});
  // prints {quotient: 6, remainder: 1}
promise.then(function (result) {
  // prints 25    
  console.log(4 * result.quotient + result.remainder);
}).catch(console.log); // not invoked

Sequel vs continuation      🗎       🗏

  • The term continuation is often misused. A continuation, in semantics parlance, represents the entire rest of the whole computation.
  • A callback only represents some computation to be done after one precise event.
  • We use “sequel” for a function attached to a promise. A sequel is not a continuation

Comparing sequel and callback      🗎       🗏

  • When a computation requires a callback, the (single) callback must be explicited before the computation starts.
  • Conversely a promise specifies a computation. Multiple sequels can be attached to that promise later or even after its resolution.

Chaining sequels      🗎       🗏

  • Promises can be chained.
  • Sequels may yield a value or a promise (more exactly a then-able object). The Promise machinery turns that result into new (successful) Promise to which new sequels can be attached.
  • When a sequel f throws an exception, the Promise machinery turns it into a new failed Promise. The failure sequels of f will then be invoked with that exception and the success sequels between f and the failure sequels will be ignored.

Chaining sequels cont.      🗎       🗏

let promise = pdivide(25, 4);
promise.then(function (result1) {
  // prints {quotient: 6, remainder: 1}
  console.log(result1);
  return result1.quotient;
})
  .then(function (result2) {
  // prints 6
  console.log(result2);
})
  .catch(console.log); // not invoked

⚙  countlinelength.1

Promise chains visualization      🗎       🗏

promise.then(s1).then(s2).catch(f3)
promise.catch(f4)
promise.then(s5).catch(f6)

ifps1s2s5then
isok   s1, s5 run
isokok  s2 runs
isokokko f3 runs
isokko  f3 runs
isok  kof6 runs
isko   f3, f4, f6 run

3.5.2  Promise composition

Promise composition

  • Promises can be orchestrated
  • Promise.all and Promise.race are the basic primitive methods.
  • Libraries such as bluebird or when offer much more methods.

Promise.all

  • Promise.all takes an array (in fact an iterable) of promises and returns a new promise, say p
  • If any of the given promises fails, then p fails.
  • If all given promises succeed, then p succeeds with the array of results.

Promise.all([promise1, promise2, ...])
  .then(function([result1, result2, ...]) { ... })

Promise.race

  • Promise.race takes an (non-empty) array (or iterable) of promises and returns a new promise, say p
  • The first given promise that resolves that is, fails or succeeds, resolves accordingly p. The other unresolved promises continue to run but do not (cannot in fact) alter the status of p.

Attention: Libraries often offer Promise.any, a variant that succeeds if one of the given promises succeeds and fails if all given promises fail. Do not confuse Promise.any and Promise.race.

Promisification

  • Node uses callbacks rather than promises.
  • Some libraries (such as bluebird ) propose to “promisify” Node functions. See also 🔍

const Promise = require("bluebird");
const fs = require("fs");
Promise.promisifyAll(fs);
fs.readFileAsync("file.js", "utf8").then(...)

3.5.3  Creation of promises

Creation of promises

  • Promises can be created ex abrupto with new Promise
  • The inner computation (say the promise body) receives two (implementation-dependent) functions to invoke after success or failure. The reject function will schedule a failure sequel(s). The resolve function is often used to schedule the invocation of the success sequel(s) (but its behavior is more complex, cf. 🔍).
  • Once resolved or rejected, the status of the promise cannot be modified.
  • The inner computation (the Promise body) starts immediately.

Example of creation of promises      👁       🗎       🗏

let p = new Promise(function (resolve, reject) {
  // compute...
  if ( ... )        // all goes well
    resolve(...);   // resolve with some result
  } else {
    reject(...);    // reject with some value
  }
});

Promise resolution      🗎       🗏

  • The inner computation (the promise body) is not a sequel, it should invoke one of resolve or reject to trigger the sequels.
  • Later calls of resolve or reject cannot modify the status of the promise: they are ignored.
  • If neither resolve nor reject is invoked, the inner computation will die silently and stay as pending indefinitely.
  • A sequel may simply return or throw to trigger its own sequels.
  • The resolve-d value from the inner computation or the return-ed value of a sequel may also be a Promise or a then-able thing (object or function). The Promise machinery (more precisely, Promise.resolve method) will convert these into promises.

then-able values      🗎       🗏

  • The Promise.resolve method will convert a (maybe then-able) value into a Promise.
  • A then-able value (object or function) possesses a then method.
  • This then method have the interface of an inner computation.

function isThenable (thing) {
  return thing !== null &&
    ( typeof thing === 'object' || typeof thing === 'function' ) &&
    typeof thing.then === 'function';
}

Promise.resolve      🗁       🗎       🗏

let p1 = new Promise(...);
p1 === Promise.resolve(p1);

let p2 = Promise.resolve(8); // fulfilled Promise
p2.then(function (result) {
  console.log(result); // prints 8
});

let pp1 = { then: (resolve, reject) => reject(9) };
let p3 = Promise.resolve(pp1); // rejected promise
p3.then(null)
  .catch((exc) => console.log('Exc' + result));
// prints Exc9

let pp2 = { then: (resolve, reject) => resolve(7) };
let p4 = Promise.resolve(pp2); // fulfilled promise
p4.then(console.log); // prints 7

Promisification implementation      🗎       🗏

  • How to turn the asynchronous callback function fs.readFile into a promise ?

fs.readFileAsync = function (file, mode = 'utf8') {
  return new Promise(function (resolve, reject) {
    fs.readFile(file, mode, function (err, data) {
      if ( err ) {
        reject(err);
      } else {
        resolve(data);
      }
    });
  });
};

Promise timeout      🗎       🗏

  • Create a promise that resolves with the result of a given promise only if that given promise is resolved (i.e., fulfilled or rejected) in less than delay milliseconds.

function atMost(promise, delay) {
  return new Promise(function (resolve, reject) {
    setTimeout(function () {
      reject(new Error('Too long'));
    }, delay);
    promise.then(resolve, reject);
  });
}

Promise.any      🗎       🗏

Promise.any takes an array of promises and return a new promise that fails if all these input promises fail. If one of the input promises succeeds then the new promise succeeds. This can be seen as the contrary of Promise.all.

Promise.any = function (promises) {
  // assume promises not empty:
  return new Promise (function (resolve, reject) {
    let done = promises.map(promise => false);
    function promiseHandler (promise, index) {
      promise.then(resolve,
        function (reason) {
          done[index] = true;
          if ( done.every(done => done) ) {
            reject("All promises fail!");
          }
      });
    }
    promises.forEach(promiseHandler);
  });
}

⚙  sequencepromises.1 ⚙  anycouple.1 ⚙  crosscheck.1

Promise style      🗎       🗏

  • Chaining promises implies to return them rather than just starting them.

p1.then(function (r1) {         p1.then(function (r1) {
  p2.then(function (r2) {                 return p2;
       ... })                   }).catch(failure1)
    .catch(failure2)              .then(function (r2) {
}).catch(failure1);                          ...
                                }).catch(failure2);

In the left part, p1 has two sequels and so has p2. These promises and their sequels are unrelated. In the right part, p1 is the head of a long series of sequels and failure of p2 will be first caught by failure1.

Prefer the writing on the right that is less involved.

Comparing promise and event      🗎       🗏

  • More than one listener or sequel may be attached to an event or a promise.
  • When an event is emitted, listeners are called synchronously. Therefore it is not possible to listen to a past event.
  • It is possible to add then or catch sequels to already resolved promises.
  • An event may be signaled more than once. A promise gets resolved only once.

3.6  Generator

3.6.1  Coroutine

Coroutine

  • JavaScript generators are inspired by coroutines.
  • A coroutine is a function that can be paused and resumed.
  • A coroutine produces a result when paused and
  • can receive information when resumed.
  • Coroutines allow to interlace computations.

3.6.2  Generator

Generator

  • Generators are produced by function*
  • Invoking a generator produces a new iterator (with additional methods) that returns objects with value and done properties.
  • The body of the iterator runs as an orphan computation.
  • The iterator is paused with yield, the argument of yield is the value returned by the iterator.
  • return sets the last value of the iterator.

Generator example

function *g (start) {
  yield start+1;  
  yield 2*start;  
  return 3*start; 
};                
let it = g(10);   
let r1 = it.next();
r1.value // 11     
r1 = it.next();
r1.value // 20
r1 = it.next();
r1.value // 30
r1.done  // true

Generator resumption      🗁       👁       🗎       🗏

  • The resumer of a paused iterator can transmit information via the next method.

function *g (start) {
  let stop = yield start+1;  
  return yield 2*stop;
};
let it = g(10);     
let r1 = it.next(); 
r1.value // 11      
r1 = it.next(21); // 21 becomes the value of stop
r1.value // 42
r1 = it.next(99); // 99 becomes the returned value
r1.value // 99
r1.done  // true

Generator methods      🗁       👁       🗎       🗏

  • In JavaScript, a computation returns a value or raises an exception.
  • Instead of resuming a paused iterator with some value, the resumer may throw a value within the iterator with its throw method.
  • The resumer can also terminate an iterator with its return method that sets the final value returned by the iterator.

Generator methods examples      🗎       🗏

function *g5 (start) {
  while ( true ) {
    try {
      start = yield start+1;
    } catch (exc) {
      start = exc;
    }
  }
}
let it = g5(0);
let r1 = it.next(); r1.value   // 1
r1 = it.throw(10);  r1.value   // 11
r1 = it.next(11);   r1.value   // 12
r1 = it.return(99); r1.value   // 99
r1.done                        // true

⚙  gfilter.1 ⚙  gmerger.1 ⚙  sumone2n.1

3.6.3  Generator delegation

Generator delegation      🗁       👁       🗎       🗏

  • yield may appear in loops, alternatives, etc. but not in internal functions.
  • yield* delegates to an iterable the production of values.

function *g3 (start) {
  while ( start < 5 ) {
    yield start++;
  }
}
function *g4 (start) {
  yield "start";
  yield *g3(start);
  yield *['s', 't'];
  return "op";
}
it = g4(0);
// values: 'start', 0, 1, 2, 3, 4, 's', 't', 'op'

3.6.4  Summary

Generator summary

  • Generators define sequential computations chopped into pieces that can be paused and then resumed.
  • An iterator keeps in memory where it was paused that is where it will be resumed with a value or where an exception would be thrown.

Comparing generator and promises      👁       🗎       🗏

  • Generators shine with promises.
  • Iterators can yield promises.
  • Helper functions (see exhaust in the example below) can manage these promises and attach to them then and catch sequels that will resume the iterator accordingly with next and throw methods.
  • This allows to interleave sequential (synchronous) code with asynchronous promises (of unknown duration)!

The following example proposes a function, named portfolio, that computes the value of this portfolio.

function exhaust (generator) {
  return new Promise (function (resolve, reject) {
    try {
      let it = generator();
      function throwError (error) {
        step(it.throw(error));
      }
      function resumeNext (v) {
        step(it.next(v));
      }
      function step (result) {
        while ( ! result.done ) {
          let promise = Promise.resolve(result.value);
          return promise.then(resumeNext).catch(throwError);
        }
        resolve(result.value);
      }
      step(it.next());
    } catch (e) {
      reject(e);
    }
  });
}

function* portfolio () {
    let wallet = JSON.parse(fs.readFileSync('wallet.json'));
    let total = 0;
    for ( name in wallet ) {
        let count = wallet[name];
        let url = `${quoteserver}${name}`;
        let body = yield http.getAsync(url);
        body = body.replace(/^\n\/\//, '');
        let value = JSON.parse(body)[0].l_cur;
        total += count * value;
    }
    return total;
}

exhaust(portfolio)
    .then((total) => {
        console.log(`total=${total}`);
    }).catch(console.log);

async and await      🗎       🗏

  • Mixing synchronous and asynchronous code is the heart of JavaScript programming
  • Generators and promises fit well but require some helper functions
  • Two new syntaxes will be added to JavaScript soon (they are not present in ES6) to get rid of these helper functions thus unencumbering code
  • The new keyword async qualifies a function definition as able to use the await keyword in its body.
  • An async function returns a Promise.
  • The body of an async function is run as an orphan computation.
  • The new keyword await expects a promise as argument and expects its resolution. If the promise is successful, await returns the resolved value. In case of a rejected promise, await throws the rejection value.
  • To sequentialize code, to confine errors with try catch eases debugging.

async function (..vars..) {..body..}
     // is somewhat similar to
function (..vars..) {
  /* function exhaust as above */
  return exhaust((function* () {
    ..body..
  }).bind(this));
}

The previous example computing the value of a portfolio can be, much simply, rewritten as:

async function portfolio () {
    let wallet = JSON.parse(fs.readFileSync('wallet.json'));
    let total = 0;
    for ( name in wallet ) {
        let count = wallet[name];
        let url = `${quoteserver}${name}`;
        let body = await http.getAsync(url);
        body = body.replace(/^\n\/\//, '');
        let value = JSON.parse(body)[0].l_cur;
        total += count * value;
    }
    return total;
}

portfolio()
  .then((total) => {
     console.log(`total=${total}`);
  }).catch(console.log);

See the next “interlude” section for other examples comparing all these various techniques.

3.7  Exercises of style

3.7.1  Interlude

Interlude: Collector      👁       🗎       🗏

  • JavaScript proposes more than one way to program.
  • To illustrate this, we propose to program a function, named concatenate, that given an array of URLs collects the titles of these web pages.
  • To simplify code, we neglect failures.
  • However we will not use any framework in order to keep code simple.

let urls = [ 'https://www.edx.org/',
             'https://developer.mozilla.org/en/' ];
function extractTitle (s) {
  let re = new RegExp('<title>(.*?)</title>');
  // Assume all urls to have a title!
  return s.match(re)[1];
}
concatenate(urls, console.log);
// { 'https://www.edx.org/':
//     "edX | Free online courses from the world's best universities",
//   'https://developer.mozilla.org/en/': 'Mozilla Developer Network' }

Collector with callbacks      🗎       🗏

function concatenate (urls, cb) {
  function conc (i, titles) {
    if ( urls.length > i ) {
      let url = urls[i], body = '';
      https.get(url, (response) => {
        response.on('data', (chunk) => body += chunk);
        response.on('end', () => {
          titles[url] = extractTitle(body);
          conc(i+1, titles);
        });
      });
    } else {
      cb(titles);
    }
  }
  conc(0, {});
}

Collector with promises      🗎       🗏

We will use this promisified version of https.get for the next examples.

function makeBodyPromise (url) {
  return new Promise((resolve, reject) => {
    let request = https.get(url, (response) => {
      let body = '';
      response.on('error', (reason) => {
        reject(reason);
        response.resume();
      });
      response.on('data', (chunk) => body += chunk);
      response.on('end', () => {
        resolve(body);
        response.resume();
      });
    });
    request.on('error', (error) => {
      reject(error);
    });
  });
}

With help of makeBodyPromise, we may now write:

function concatenate (urls) {
  let titles = {};
  return Promise.all(urls.map(url =>
           makeBodyPromise(url).then((body) =>
              titles[url] = extractTitle(body))));
    .then(() => titles);
}

Collector with generators      🗎       🗏

function concatenate (urls) { 
  function exhaust (generator) {
    return new Promise (function (resolve, reject) {
      try {
        let it = generator();
        function throwError (error) {
          step(it.throw(error));
        }
        function resumeNext (v) {
          step(it.next(v));
        }
        function step (result) {
          while ( ! result.done ) {
            let promise = Promise.resolve(result.value);
            return promise.then(resumeNext).catch(throwError);
          }
          resolve(result.value);
        }
        step(it.next());
      } catch (e) {
        reject(e);
      }
    });
  } 
  function *conc () {
    titles = {};
    for ( let url of urls ) {
      let title = extractTitle(yield makeBodyPromise(url));
      titles[url] = title;
    }
    return titles;
  }
  return exhaust(conc);
}

Collector with async/await      👁       🗎       🗏

This example requires node-v8 to be run, node-v6 does not support async/await.

async function concatenate (urls) {
  let titles = {};
  for ( let url of urls ) {
    try {
      let body = await makeBodyPromise(url)
      titles[url] = extractTitle(body);
    } catch (e) {
      titles[url] = undefined;
    }
  }
  return titles;
}
// concatenate(urls).then((titles) => titles);

Another variant with a lot more concurrency:

async function concatenate (urls) {
  let bodies = await Promise.all(urls.map(makeBodyPromise));
  let titles = bodies.map(extractTitle);
  let result = {};
  urls.forEach((url, index) => {
    result[url] = titles[index];
  });
  return result;
}
// concatenate(urls).then(titles => titles);

3.8  Worker

3.8.1  Worker

Worker

  • Since JavaScript is organized around an event loop, long computations must be chopped in small pieces for the sake of reactivity.
  • Another solution is to use Workers.
  • A worker is a separate and independent task (or process in Unix parlance) that runs concurrently to the main event loop.
  • A worker is almost similar to a program running on another machine with Internet-based relationship with its creator.
  • Unfortunately, the interface for Workers running in Node is different from the interface for Workers running in browsers though some libraries may mask these differences.

Workers features

  • The main questions are
    • How to create a worker ?
    • What code the worker will run ?
    • What is the global environment of the worker ?
    • How to send information to a worker ?
    • How the worker can send back information to its creator ?
    • How its creator can terminate a worker ?
    • How a worker can terminate itself ?

Worker visualization

Worker common workflow

  • When created the worker has to get the code it will run. Meanwhile the creator is not blocked.
  • The worker or the creator may send information to the other whenever they want, even repeatedly.
  • After setting the appropriate handler, the worker and the creator can receive information from the other.
  • Workers or creator may initiate the first transmission of information.
  • Worker or creator are not compelled to obey a request-response sequence.
  • The worker may terminate itself or be terminated by its creator.

3.8.2  Workers in browsers

Workers in browsers

// Create a worker and interact with it: 
let w = new Worker('url/script.js');
w.onmessage(function (result) {
  ... // use result.data
});
w.postMessage(someInput);
w.terminate(); // to terminate the worker

// In the worker i.e., in url/script.js
self.onerror = function (error) { ... };
importScripts('url/otherscript.js');
self.onmessage = function (input) {
  ... // use input.data
  postMessage(someResult);
};
self.close(); // to terminate the worker

Web workers global environment

  • The global environment of a worker is the value of self
  • self is not a copy of window. In particular, the DOM is not present.
  • self allows Ajax requests, WebSockets, access to local storage, timeout handlers, writing to console, etc.
  • self can be enriched with other scripts loaded with importScripts

Information passing

  • A value passed to or from a worker is serialized into a string, then transmitted and finally deserialized into a value.
  • Values are copied not shared!
  • Not all values can be transmitted! Closures, cyclic structures, prototype, errors may not be transmitted depending on implementations.
  • But JSON values can safely be transmitted.

3.8.3  Workers in Node

Workers in Node      👁       🗎       🗏

  • Workers in Node.js are inspired by Unix processes.
  • fork starts a new process that runs the same code, again from its beginning. What changes is the isMaster and isWorker boolean properties.
  • Server sockets opened before fork are shared with workers. This allows servers to balance the load on their workers.

const cluster = require('cluster');
console.log("*printed twice");

if (cluster.isMaster) {  // code for master
    cluster.on('online', (_worker) => {
        console.log(`Master, slave is now online`);
    });
    cluster.on('message', (worker, msg) => { 
        // process messages from workers
        if ( msg.kind === 'ready' ) {
            // worker is now ready
            worker.send("Hi slave");
        } else {
            console.log(`Master: received`, msg);
            if ( msg.counter > 2 ) {
                worker.send({kind: 'stop'});
            } else {
                worker.send("Hi again");
            }
        }
    });
    cluster.on('exit', function (worker, code, signal) {
        console.log(`Master exiting`);
        cluster.disconnect();
    });
    // Create worker:
    cluster.fork();

} else {           
    // code for slave
    let counter = 0;
    process.on('message', function (msg) {
        // process message from master:
        if ( msg.kind === 'stop' ) {
            console.log(`Slave: stop`);
            process.disconnect();
        } else {
            console.log(`Slave: received ${msg}`);
            // send to master
            process.send({kind: 'time', counter: ++counter});
        }
    });
    process.send({kind: 'ready'}, () => {
        console.log(`Slave: sent ready to master`);
    });
}

Node workers global environment      🗎       🗏

  • The global environment of a Node worker is the value of global (as every other Node process).
  • The process object holds the functions and the event listeners needed to communicate with the master or workers.

3.8.4  Workers use cases

Workers use cases

  • To make use of another CPU for a CPU-intensive computation.
  • For a computation of unknown duration
    • Start a worker to poll regularly a remote server and detect if some information is present (for instance, a stock price) and satisfies some condition.
    • Start a farm of workers draining a queue of tasks sent by the master
      • via files (to be importScript-ed)
      • via modules (to be require-d)
      • via strings (to be eval-uated)
      • or via Blobs

⚙  decrypt.1

Workers unification

3.9  Summary

3.9.1  Summary

Conclusions

  • JavaScript offers a lot of constructions. Node predefined libraries uses callbacks and events most of the time.
  • Events must be listened before being emitted or they are lost. Promises can acquire new sequels even after resolution.
  • Events may be repeatedly emitted, promises get resolved only once.
  • Callbacks impose sequentiality, Promise.all offers more potential concurrency.
  • Workers cannot share environments but may run truely concurrently.
  • Generators interleave synchronous and asynchronous computations.

4  Distribution (Week 4)

4.1  Foreword

4.1.1  Foreword

Foreword

Envisioning separately servers and clients might not yield interesting applications. Nowadays, applications are diffuse that is, they must encompass multiple clients and multiple servers cooperating together to provide interesting functionalities.

Depending on the power, battery level, memory size of phones, tablets or computers, some computations can be performed on servers or on clients and this can be decided even dynamically.

The client-server paradigme is asymetric, clients can request servers but servers can only answer. Web socket is a new API that allows clients to open bi-directional tunnels towards servers. Once established, this connection can be used by both ends to communicate with the other endpoint. For instance, with web sockets, servers can decide, whenever they want, to push alerts to all connected clients.

Here are the topics addressed below:

  • Client-server model
  • The HTTP protocol
  • Caching
  • Model-View-Controller
  • Servers architecture
  • REST style
  • Multiple MVC
  • WebSocket
  • Security

4.2  Client-Server relationships

4.2.1  Client-Server model

Client-server exchanges

  • The traditional client-server model is an asymetric model where
    • clients request servers
    • servers respond to clients
  • At the beginning, servers respond with HTML pages that replace the original page.
  • Servers evolve and respond with dynamic HTML that is HTML pages with JavaScript code that can operate upon the DOM (Document Object Model) that is, the reified representation of the HTML page.
  • Later XMLHttpRequest allow browsers to ask for dynamic HTML page fragments, graft them in the current DOM and link-edit new JavaScript code within the current JavaScript global environment.
  • Then, requests also return XML document or JSON data interpreted by JavaScript code and transformed into HTML for visualization.
  • However the underlying technology (TCP sockets) is symetric. Browsers evolve and Web sockets offer symetry: servers can push information to connected client(s).

Client-server exchange types

  • A click on an anchor <a href> or an image <img src> fetches some data that will be displayed (as a new page) by a default browser’s callback according to the MIME type of the data.
  • a <script> element fetches some JavaScript code that will be evaluated in the current global environment.
  • an XMLhttpRequest fetches some data that will be processed by some user-specified callback in the current global environment.

Client-server exchange formats

  • XML (HTML reduced to its structural skeleton, independent of any visualization information) was the format of choice. However, XML needs parsers thus needs CPU, is verbose thus needs bandwidth. Conversely, XML is ruled by grammars (XMLSchema, RelaxNG) that forbids badly formatted information thus shortening programs (format error handling is no longer necessary).
  • JSON is JavaScript Object notation with numbers and strings, hashtables and arrays. JSON is lighter (requires less CPU and bandwidth) and needs much simpler parsers. However formating is more versatile and more error-prone.

In both cases, data is analyzed, interpreted by JavaScript code, is used to update local information and often transformed into new HTML grafted in the current DOM.

In a single page application (today’s trend), API are written in JavaScript and JSON is far easier to process than XML.

⚙  xmljson.1

4.3  HTTP

4.3.1  HTTP

HTTP and servers

  • HTTP is the transport protocol used for nearly every exchange of information
  • The requestMethod (complemented with specific HTTPparameters) determines how the network behaves: how the request is transported or cached.
  • The format of an HTTP request structures how servers answer. The problem of a server is to determine what code is triggered by an HTTP request ?

HTTP GET or HEAD request

  • HTTP is more than a transport protocol since it also brings semantics in the exchanges of information.
  • HTTP GET or HEAD request get resource named by an URL
  • To serve this request should not modify the server, therefore and unless forbidden, the response may be cached by the network
  • To this URL may be adjoined some query parameters to further refine the query (for instance, list books from 10th to 20th)
  • The client may also specify HTTP parameters to describe itself (for instance, I accept only French, only PNG images but I support bzip2 compression)
  • HEAD request is similar to GET however the body of the response to a HEAD request is empty. This allows to get the meta-information accompanying the resource such as its type or size.

HTTP POST or PUT request

  • Upload some resource on a server
  • The body of the request is the resource to upload. An empty line separates the HTTP parameters from the body. The length of the resource is specified in the Content-Length HTTP parameter. If the resource is compressed, the kind of compression should be specified in Content-Encoding HTTP parameter.
  • PUT specifies the name of the resource that is uploaded
  • POST asks the server to name the resource that is uploaded, the server’s answer will contain that new name in the Location HTTP parameter.
  • POST is also used with HTML forms.
  • POST modifies the server, the network should never cache this request
  • PUT is idempotent, uploading again the same resource does not change the server.

HTTP DELETE request

  • Remove some resource held in the server
  • DELETE is idempotent, deleting an already deleted resource is tolerated.
  • POST(Create), GET(Read), PUT(Update), DELETE are the four commands providing a CRUD application.

HTTP response

  • The most important information in a response is the code , a human readable explanation of the code follows that code. It is important to use the appropriate code.
  • The 2xx series is OK among which (200 is cacheable):
    • 200 the generic OK,
    • 201 means “created” (in response to PUT or POST)
  • the 3xx series is for redirection (the response often contains a Location HTTP parameter) among which (301 is cacheable):
    • 301 moved permanently
    • 302 moved temporarily
    • 304 not modified (your copy is still valid)
    • 303 GET an equivalent resource elsewhere
    • 307 redirect your request elsewhere (with same request method)
  • the 4xx series is for client errors among which:
    • 400 the generic bad request
    • 403 forbidden (authentication required)
    • 404 resource not found
    • 406 cannot convert the resource into a representation accepted by the client
  • the 5xx series is for server errors among which:
    • 500 generic internal server error
    • 503 service unavailable

4.3.2  HTTP and caching

Caching files

  • Caching specification evolve from HTTP/1.0 to HTTP/1.1, we stick to the latter.
  • If the response is a file, the date of the last modification of that file can appear in the HTTP response parameter Last-Modified
  • The client takes notice and may ask for that file again with an additional HTTP request parameter If-Modified-Since.
  • The response code may be 304 Not modified if the file is still the same (the body of the response is then empty thus sparing bandwidth) or 200 (and a body containing the new content of the file).
  • Of course, you can use this re-validation mechanism for any resource for which you can track the last modification date.

Caching with expiration

  • The previous mechanism requires clients to re-validate that is, check if their cached copy of a resource is still valid. This is wasteful.
  • The server may serve a resource with an HTTP response parameter Expires telling the client that the resource can be cached till the expiration date.
  • Pay attention to timezones!
  • Instead of an expiration date, the server may specify an expiration duration (in seconds) with Cache-controlmax-age=. This specifies that the resource will not change in the next seconds.
  • The server can also impose clients to re-validate their cached resources with Cache-controlno-cache

⚙  clocktick.1

4.4  Model-View-Controller

4.4.1  MVC

Model View Controller - Basics

  • Invented by Trygve Reenskaug in Smalltalk-76 (that is, before 1976)
  • MVC is a software pattern that separates input, state and output.
  • The model that is, the state of the application.
  • Controllers interpret the inputs (events, clicks, gestures, etc.) and convert them into actions transforming the model.
  • Views are produced from the model and delivered as the result of an input.

Model View Controller - Components

  • The three components are independent
  • and can be developped independently.
  • Alternate views can be developped (CSV instead of HTML, JSON instead of SVG, etc.)
  • New gesture, new types of input can be introduced with new controllers
  • And, of course, the model can be re-implemented but must respect the model API.

Model View Controller - Variants

  • MVC is a theoretical model with many variants
  • A controller should have a way to report errors
    • error from the model
    • or from the controller
  • Views should use the model API
    • model can help and provides only the changes (for instance, changing CSS usually does not require to re-draw the entire DOM)
  • Views may have an internal state (holding user’s preferences for instance)
  • Controllers may have an internal state (collecting enough information before making the appropriate actions on the model)
  • Model is usually persisted in a database.

4.4.2  Servers architecture

Servers architecture

  • Inputs are HTTP requests (and HTTP responses if the server also acts as a proxy)
  • A framework (such as express , meteor , etc.) has to determine which code will process the HTTP request
  • Quite often, the pathInfo is first considered then the requestMethod to determine which function (or method) to invoke. Tables describing this association is usually named routes.
  • Parameters of all types (HTTPparameters, queryString, etc.) are usually provided via hashtables or variables.
  • The body is decoded along the Content-Type HTTPparameter
  • An empty Response object is also provided that the elected function has to fill (return code, body). Once the body begins to be emitted, no additional HTTP header can be adjoined!
  • HTML response body is often generated with templates. Preferred client’s languages (Accept-Language) may be taken into account.
  • The framework frees you of many boring tasks such as describing the available features of the underlying server (compression, language, MIME-types supported, etc.) as well as which protocol version (HTTP/1.0 or HTTP/1.1) or which compression to use when emitting the body, etc.

4.4.3  REST style

REST style

  • RPC (remote procedure call) or RMI (remote method invocation) use URL containing verbs. The client asks the server to perform some operation specified as a verb.

    
    https://server/login?name=John&token=123456789qwertyui
    https://server/rotateLeft?photo=34
    http://server/book?isbn=9780521562478&format=json
    

  • REST (Representational state transfer created by Roy Fielding in 2000) names resources with URLs that are operated with HTTP request methods: POST(Create), GET(Read), PUT(Update), DELETE.

    
    GET https://server/photo/34?rotated=-90
    GET http://server/book/9780521562478/
        with Accept: application/json
    POST http://server/book/9780521562478/coverpage 
    

  • Some patterns are often used to name resources. Naming collections, one item of a collection, getting or setting an attribute of an item.

    
    /elements
    /element/id
    /element/id/attribute
    /element/id/attribute/newvalue
    

  • Many frameworks make easy to use these patterns.

⚙  counter.1

4.4.4  Multiple MVC

Multiple concurrent MVC

  • The server is organized around a MVC model that reacts to HTTP requests or responses, SQL responses, etc.
  • The client is organized around a MVC model that reacts to user’s interactions, HTTP responses, etc.
  • The problem is to make these two (at least) MVC cooperate
  • via HTTP or WebSocket (and probably JSON messages)

4.5  WebSocket

4.5.1  WebSocket

The WebSocket protocol

  • WebSocket (WS) is a bidirectional protocol between client and server.
  • A connection started with HTTP may be upgraded to WS (and similarly from HTTPS to WSS).
  • The WS protocol is accessed, in the browser, via a JavaScript API (somewhat similar to the worker API)
  • Messages are strings (transmitting JSON objects needs JSON.stringify) but might also be binary ArrayBuffers.
  • Some frameworks such as socket.io are built on top of WS.

// Browser side:  
let myWebSocket = new WebSocket("ws://www.websockets.org");

myWebSocket.onopen = function (evt) {
  alert("Connection open ..."); };
myWebSocket.onmessage = function (evt) {
  alert( "Received Message: " + evt.data); };
myWebSocket.onclose = function (evt) {
  alert("Connection closed."); };      

myWebSocket.send("Hello WebSockets!");
myWebSocket.close();

  • Use a npm module such as ws for instance on the server.
  • Browsers provide their own WebSocket class.

// Server side with ws module
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });

wss.on('connection', function connection (ws) {
  ws.on('message', function incoming (data) {
    // Broadcast to everyone else. 
    wss.clients.forEach(function each (client) {
      if (client !== ws && client.readyState === WebSocket.OPEN) {
        client.send(data, function (error) {
          if ( error ) {
            console.log(`cannot send ${error}`);
          }});
      }
    });
  });
});

4.6  Security

4.6.1  Security

OWASP

  • Open Web Application Security Project publishes regularly the top 10 security problems that are:
    1. Injection
    2. Broken Authentication and Session Management
    3. Cross-Site Scripting (XSS)
    4. Broken Access Control
    5. Security Misconfiguration
    6. Sensitive Data Exposure
    7. Insufficient Attack Protection
    8. Cross-Site Request Forgery (CSRF)
    9. Using Components with Known Vulnerabilities
    10. Underprotected APIs
  • The first three items are in that position for decades!
  • Read the document and be aware of existing solutions
  • Use frameworks that are safe with respect to these problems.

4.6.2  Same Origin Policy

Same Origin Policy

  • A browser decodes, interprets or runs sequences of bytes coming from everywhere (images, sounds, scripts, etc.): this is dangerous!
  • Same Origin Policy (SOP) lessens the risks
  • Received cookies can only be sent to the sending host (or to another host of a broader domain if the cookie’s domain is specified). A secure property reserves cookies to https only. However document.cookie is a public property.
  • When receiving a page from an origin (specified as protocol://server:port/), complete access is granted to all URLs with that same prefix.
    • SOP prevents mixed content where a page contains both http and https resources (mostly images) see also the HTTPparameter Upgrade-Insecure-Request or Strict-Transport-Security.
    • Within browsers, you cannot assume to read properties of JavaScript objects created from other origin. See also the document.domain property.
    • For XMLhttpRequests: PUT and DELETE requests cannot be sent to other origins but POST and GET can be sent elsewhere but without custom HTTPparameters.
  • However scripts (resp. images, css, frames, iframes) can be downloaded and run (resp. displayed) from everywhere (to allow CDN for intance).

4.6.3  Cross-origin resource sharing

Cross-origin resource sharing

  • Cross-origin resource sharing (CORS) is a mechanism to allow XMLhttpRequests to bypass SOP if the other server agrees. It allows all request methods, custom HTTP request parameters.
  • When the browser detects a SOP violation, it sends a “pre-flight” OPTION request to the targeted server with information on the Origin of the embedding page, the intended request method (with Access-Control-Request-Method and the intended custom HTTP request parameters (with Access-Control-Request-Headers)
  • The server analyzes these three HTTP request parameters and if it agrees, it responds with Access-Control-Allow-Origin HTTP response parameter.
  • The Access-Control-Allow-Origin: * means that the resource can be accessed publicly. Otherwise Access-Control-Allow-Origin: origin only allows the access to the requesting origin. Additional HTTP response parameters precise which request methods, which headers are allowed and for how long.
  • More importantly the Access-Control-Allow-Credentials tells whether the server will accept cookies or not.
  • The browser compares the characteristics of the initial request and the constraints returned by the server and decides if sending the initial request is appropriate.

4.6.4  API protection

API protection

  • Use of API should be protected from pirats but also from legal users
  • Use HTTPS to mask the details of authentication (mainly API key and other secure cookies or tokens)
  • Implement applicative security and check if user U has the right to run service S on object O. For instance, a user may change its name but not others’ name. However it is a simple deduction that if changing its own name triggers a PUT request such as https://.../user/123456/?newname=XX then one may also try the same request with other user identifiers!

4.7  Exercises of style

4.7.1  Interlude

Interlude: Chain

This is a small example of a chain of related objects ending in a row in an SQL database. Modifying an object in the browser (1), transfers the modification (2) to the corresponding object hosted on the server (via a REST API on HTTP) which in turn records it (3) in the database and broadcasts the modification via WebSocket (4) to all connected clients (5).

This code uses many features of JavaScript (closures, objects, events, promises) and two protocols (HTTP and WebSocket). It does not use any framework (ORM, WebServer, WebSocket, etc.) in order to show, focus and explain the essence of the code disembarrassed of extra whistles or bells. Therefore, besides the usual Node modules (http, fs, events and url), it only requires two extra modules:

  • sqlite3 to access an SQL database (here SQLite) and
  • ws to manage WebSockets on the server side.

We will explain this code and its main features below. We will also propose some extensions but you may also search for other (more mature) Node modules dedicated to this same goal.

The explanations will successively present:

  • src/dbobject.js a tiny ORM (Object Relational Mapping) that put JavaScript objects in correspondance with rows of SQL tables.
  • src/webapi.js that offers an HTTP REST API to operate on dbobjects with GET, POST, PUT and DELETE requests.
  • src/browserobj.js a client library that offers browser objects linked to remote dbobjects.
  • src/wsapi.js a WebSocket server that broadcasts modifications to dbobjects to their related browser objects.

The whole code as well as a small webapp (defined in file chain.js) is available with the djs-chain npm module. Fetch that code to make your own experiments.

4.7.2  Interlude - dbobject

dbobject

This file defines four classes:


  -- Class --            -- Public methods --
  DB                     createTable, persistAll, getTables, close
     DBsqlite3  inherits from DB
  DBTable                insert, get, all
  DBObject               remove
     one different subclass for every DBTable instance

Here follow the main design decisions:

  • Every row is reified as a JavaScript object with properties named after the columns of the containing table.
  • When an object is reified, only one copy of it exists in JavaScript memory and this copy is always uptodate.
  • All objects coming from the same table are instances of the same class, a subclass of DBObject. This allows for custom methods specific to a single table. Not duplicating properties is also a goal.
  • When a reified object is modified, a promise is generated to transfer the modification into the database. All these promises are sequentialized to respect the modification order.
  • Methods on table: insert, get and all return promises.
  • The code is unsafe and only partial, many useful methods could be added and sanity of many parameters should be checked.

The next figure shows the relationships between a row in an SQL table from the database and a JavaScript object (value of the joe variable) and its chain of classes and prototypes.