Monday, September 7, 2015

TypeScript All Things

On following Angular controller code, the second parameter Item looks odd, it has no type.

And the only way we can get TypeScript's autocomplete goodness on that Item service is to explicitly assign the type on Item service's $promise's callback argument. We cannot get autocomplete support on customQuery method.

constructor(
    product: Domain.Product, 
    Item, 
    $stateParams, 
    appWide: SharedState.AppWide) 
{

    appWide.title = "Products";

    this.product = product;

    var tag = $stateParams.tag === undefined ? "" : $stateParams.tag;

    product.searched = tag === "" ? "" : "tag:" + tag.replaceAll("--", " ");

    product.search = () => {
        this.searched = product.searched;

        Item.customQuery({keywords: this.searched})
            .$promise.then((items : 
                Dto.SearchDto
                ) => {
                this.itemSearched = items.matches;
                product.tags = items.tags;
                product.locations = items.locations;
            });
    }; // search

    product.search();

}

Definition of Dto.SearchDto:
module Dto {
    export class SearchLocationDto {
        locationId: string;
        locationName: string;
    }

    export class SearchItemDto {
        itemId : number;
        itemTitle: string;
        itemDescription: string;
        itemTags: string;
    }

    export class SearchDto {
        matches: SearchItemDto[];
        tags: string[]  ;
        locations: SearchLocationDto[];
    }
}

Definition of Dto.IItemReturnedDto. This is the type of save's then's success callback argument. Just a primary key
module Dto {
    export interface IItemReturnedDto {
        item_id : number;
    }
}


Then on module initialization, objects are just returned typeless:
var mod:ng.IModule = angie.module('niceApp', ['ui.router', 'ngResource', 'scs.couch-potato', 'ngFileUpload']);

mod.factory('Item', ['$resource', ($resource:angular.resource.IResourceService) => {

    return $resource("/api/item-secured/:theItemId", null, {
        customQuery: {method: 'GET', isArray: false}
    });

}]);


To improve the above, provide type on a service derived from Angular's $resource. Following is the interface definition of customized angular resource method customQuery.
module InhouseLib {

    export interface ISearchParameter {
        keywords: string;
    }

    // the returned when doing save on resource is Dto.IItemReturnedDto
    export interface ISearchResource extends ng.resource.IResourceClass<ng.resource.IResource<Dto.IItemReturnedDto>> { 
        customQuery(ISearchParameter) : angular.resource.IResource<Dto.SearchDto>;
    }
}

Then on module initialization, use the type defined above.
mod.factory('Item', ['$resource', ($resource:ng.resource.IResourceService) : InhouseLib.ISearchResource => {


   var customQueryAction : ng.resource.IActionDescriptor = {
       method: 'GET',
       isArray: false
   };

   var r = <InhouseLib.ISearchResource> $resource ("/api/item/:theItemId", null, {
       customQuery: customQueryAction
   });

   return r;

}]);


Now on controller constructor, add the type for Item (line 3 on code at the bottom). Doing that, the autocomplete for customQuery on Item service shall pop-up upon typing the dot character.




And also, we can also now eliminate (on line 20) the type (Dto.SearchDto) of the callback argument items:

constructor(
     product: Domain.Product, 
     Item: InhouseLib.ISearchResource, 
     $stateParams, 
     appWide: SharedState.AppWide) 
 {

    appWide.title = "Products";

    this.product = product;

    var tag = $stateParams.tag === undefined ? "" : $stateParams.tag;

    product.searched = tag === "" ? "" : "tag:" + tag.replaceAll("--", " ");

    product.search = () => {
        this.searched = product.searched;

        Item.customQuery({keywords: this.searched})
            .$promise.then(items => {
                this.itemSearched = items.matches;
                product.tags = items.tags;
                product.locations = items.locations;
            });

    }; // search

    product.search();

}

And yet the IDE is smart to know the type and it gives an autocomplete when you type the variable name(items) and dot symbol. TypeScript and WebStorm IDE are nice.

Sunday, September 6, 2015

Custom Basic Authentication and Authorization with AngularJS, Node Express, TypeScript, Massive DAL and PostgreSQL

Design:
    • On successful login, the server shall send back the authentication header that will be saved to angular's $http pipeline
    • Authorization is roles-based
    • Member's role(s) are stored as text array in the database

Prerequisites:
    • Data Access Layer node module
        ○ npm install massive
    • Basic Authentication node module
        ○ npm install basic-auth



I'll go owl on this and just do a code dump and briefly explain each part and their relation to other parts.


What will be created:
    
    • Login screen:
        1. /public/app-dir/Login/Template.html
        2. /public/app-dir/Login/Controller.ts
        
    • Login and Response DTOs shared by client-side and server-side
    
        3. /shared/dto/LoginDto.ts
        4. /shared/dto/LoginResponseDto.ts
        
    • Authentication service for Angular
        5. /public/inhouse-lib/AuthService.ts
    
    • Authentication node service
        6. /api/member.ts
        7. /db/loginVerify.sql

    • Authorization for node REST APIs
        8. /server/CanBe.ts
        9. /db/verifyRoleAccess.sql

    • Member table
        10. server/ddl.sql

    
What need to be changed:
        11. Nodejs's app.ts
            i. Add initialization of massive DAL
            ii. Add authentication url to node
            iii. Authorize a REST API




1. public/app-dir/Login/Template.html
<style>

    .block label { display: inline-block; width: 140px; text-align: left; }

</style>

<div class="col-md-2 col-md-offset-5">
    <form ng-submit="c.login()">
        <div class="block">
            <label>Username</label>
            <input type="text" ng-model="c.username"/>
        </div>

        <div class="block">
            <label>Password</label>
            <input type="password" ng-model="c.password"/>
        </div>

        <div class="block">
            <label></label>
            <input type="submit" value="Login"/>
        </div>

    </form>
</div>

2. public/app-dir/Login/Controller.ts
///<reference path="../../lib-inhouse/doDefine.ts"/>
///<reference path="../../../typings/angularjs/angular.d.ts"/>
///<reference path="../../lib-inhouse/AuthService.ts"/>
///<reference path="../../../typings/sweetalert/sweetalert.d.ts"/>
///<reference path="../../shared-state/AppWide.ts"/>

module App.Login {

    export class Controller {


        username:string;
        password:string;


        // Dependency-injected sweet alert, so it can be easily mocked

        constructor(public authService:InhouseLib.AuthService, public $http:ng.IHttpService, public $state:any,
                    public TheSweetAlert:SweetAlert.SweetAlertStatic,
                    public appWide:SharedState.AppWide) {
        }


        login():void {


            this.authService.verify(this.username, this.password)
                .then(success => {
                    this.appWide.isUploadVisible = true;
                    this.$state.go('root.app.home');
                }, error => {
                    this.appWide.isUploadVisible = false;
                    this.TheSweetAlert(error.status.toString(), "Not authorized", "error");
                });


        }

    }
}


doDefine(require => {

    var mod:angular.IModule = require('eat');


    require('/shared/dto/LoginDto.js');


    mod["registerController"]('LoginController', ['AuthService', '$http', '$state', 'TheSweetAlert', 'AppWide',
        App.Login.Controller]);

});


authService.verify line 27, is where the setting and clearing of Basic Authentication information to Angular's $http pipeline happens, authService.verify returns a promise if the authentication is successful, if the user is not authenticated it goes to the promise's error callback parameter.


public/lib-inhouse/doDefine.ts
///<reference path="../../typings/requirejs/require.d.ts"/>

// the doDefine will be disabled during unit test

function doDefine(func: (cb) => any) {

    define(func);

}

3. shared/dto/LoginDto.ts
///<reference path="../../typings/node/node.d.ts"/>


module Dto {

    export class LoginDto {

        username: string;
        password: string;

    }

}



4. shared/dto/LoginResponseDto.ts
///<reference path="../../typings/node/node.d.ts"/>

module Dto {

    export class LoginResponseDto {

        isValidUser: boolean;

        theBasicAuthHeaderToUse: string;

    }

}

The theBasicAuthHeaderToUse is the header that will be received by Angular and shall be assigned to Angular's $http pipeline. Its format is:

Basic base64encodeOf(username + ': ' + password)

Sample format:
Basic VGhvbSBZb3JrZTpjcmVlcA==


5. public/lib-inhouse/AuthService.ts
///<reference path="../../typings/angularjs/angular.d.ts"/>
///<reference path="../../shared/dto/LoginDto.ts"/>
///<reference path="../../shared/dto/LoginResponseDto.ts"/>

module InhouseLib {

    export class AuthService {


        constructor(public $http:ng.IHttpService, public $q:ng.IQService) {
        }

        verify(username:string, password:string):ng.IPromise<ng.IHttpPromiseCallbackArg<Dto.LoginResponseDto>> {

            var deferred = this.$q.defer();

            var u = new Dto.LoginDto();
            u.username = username;
            u.password = password;



            this.$http.post<Dto.LoginResponseDto>('/api/member/verify', u)
                .then(success => {


                    this.$http.defaults.headers["common"]["Authorization"] = success.data.theBasicAuthHeaderToUse;

                    // This goes to then's success callback parameter
                    deferred.resolve(success);

                }, error => {

                    this.logout();

                    // this goes to then's error callback parameter
                    deferred.reject(error);

                });

            return deferred.promise;

        }

        logout(): void {
            delete this.$http.defaults.headers["common"]["Authorization"];
        }

    }

}



6. Authentication module. Do the base64 encoding of Basic username:passwordHere at the server.

api/member.ts
/// <reference path="../typings/express/express.d.ts"/>
/// <reference path="../typings/extend/extend.d.ts"/>
/// <reference path="../shared/dto/LoginDto.ts"/>
///<reference path="../shared/dto/LoginResponseDto.ts"/>


import express = require("express");

import extend = require('extend');


export function verify(req:express.Request, res:express.Response, next:Function):any {


    var app:express.Application = req["app"];
    var db = app.get('db');


    var loginDto = <Dto.LoginDto> req.body;
    console.log(loginDto.username);

    var loginResponseDto = <Dto.LoginResponseDto>{};

    db.loginVerify([loginDto.username, loginDto.password], (err, result) => {


        if (result.length == 0) {

            loginResponseDto.isValidUser = false;
            loginResponseDto.theBasicAuthHeaderToUse = "Ha! Are you expecting a returned value?!";
            res.status(401).json(loginResponseDto);

        }
        else {

            var member = result[0];
            var salted_password = member.salted_password;

            loginResponseDto.isValidUser = true;
            loginResponseDto.theBasicAuthHeaderToUse = 
                "Basic " + new Buffer(loginDto.username + ":" + loginDto.password).toString("base64");
            res.json(loginResponseDto);
        }

    });

}



7. db/loginVerify.sql

This is called by the Authentication module.

For a primer of bcrypt mechanism: http://www.ienablemuch.com/2014/10/bcrypt-primer.html

select member_name, salted_password
from member
where member_name = $1 and salted_password = crypt($2, salted_password);

This is where the authorization happens.


8. Authorization module

server/CanBe.ts
/// <reference path="../typings/express/express.d.ts"/>

import express = require('express');

var basicAuth = require('basic-auth');


export function accessedBy(roles:string[]):express.RequestHandler {
    return accessedByBind.bind(undefined, roles);
}


function accessedByBind(roles:string[], req:express.Request, res:express.Response, next:Function): any {

    function unauthorized(): any {
        res.set('WWW-Authenticate', 'Basic realm=Authorization Required');
        return res.sendStatus(401);
    }


    // === accessedByBind starts here ===

    var app:express.Application = req["app"];
    var db = app.get('db');


    var user = basicAuth(req);

    if (!user || !user.name || !user.pass) {
        return unauthorized();
    }

    var username = user.name;
    var password = user.pass;

    db.verifyRoleAccess([username, password, roles], (err, result) => {

        if (err == null) {
            var isAllowed = result[0].is_allowed;

            if (isAllowed)
                return next();
            else {
                return unauthorized();
            }
        }
        else {
            return res.status(500).json({customError: 'Unknown Error'});
        }

    });


}



9. db/verifyRoleAccess.sql

This is called by the CanBe Authorization module.

select exists(

select null
from member
where
member_name = $1 and salted_password = crypt($2, salted_password)
and roles && $3 -- check if $3 (arrayOfRoles) passed from canBe.accessedBy(arrayOfRolesHere) is in member.roles.

) as is_allowed;



10. server/ddl.sql
create table member
(
    member_id serial primary key,
    member_name citext not null unique,
    salted_password text not null, -- using bcrypt
    roles text[]
);

insert into member(member_name, salted_password, roles) values
('Thom Yorke', crypt('creep', gen_salt('bf')), '{"rockstar"}')


11. app.ts
var massive = require('massive'); // DAL
import memberApi = require('./api/member'); // authentication 
import canBe = require('./server/CanBe'); // authorization


var massiveInstance = massive.connectSync({connectionString: 'postgres://postgres:yourPasswordHere@localhost/commerce'});
app.set('db', massiveInstance);

app.post('/api/member/verify', memberApi.verify);

// unsecured API
app.get('/api/item', itemApi.search);

// just insert canBe.accessedBy before the itemApi.search RequestHandler REST API to secure the REST API
app.get('/api/item-secured', canBe.accessedBy(['rockstar']), itemApi.search); 

UPDATE:

Some code above has Cargo Culting in it, it uses $q.defer unnecessarily. Haven't properly learned the promise's fundamentals before :) Read: https://www.codelord.net/2015/09/24/%24q-dot-defer-youre-doing-it-wrong/

Don't marshal when you can cast

var loginDto = <Dto.LoginDto> {};
extend(loginDto, req.body); // marshal the req.body dictionary to strongly-typed object
console.log(loginDto.username);

The code above is inefficient, extending is an expensive process. The above could be simply rewritten as:

var loginDto = <Dto.LoginDto> req.body;
console.log(loginDto.username);


Imagine the nodejs code above if done with statically-typed framework such as ASP.NET MVC, the values from Request.Form (same as nodejs's req.body) need to be marshalled to DTO object every time a request is made, a bit inefficient.


You can make something efficient and be efficient (autocomplete support) when coding with TypeScript.

Thin Angular Controller

An example of caller of Authentication API:

App.Login.Controller.ts:
module App.Login {

    export class Controller {


        username:string;
        password:string;

        // Dependency-injected sweet alert, so it can be easily mocked
        constructor(public authApi:InhouseLib.AuthApi, public $http:ng.IHttpService, public $state:any,
                    public TheSweetAlert:SweetAlert.SweetAlertStatic,
                    public appWide:SharedState.AppWide) {
        }


        login():void {


            this.authApi.verify(this.username, this.password)
                .then(success => {
                    console.log(success.data.isValidUser);
                    this.appWide.isUploadVisible = true;
                    this.$http.defaults.headers["common"]["Authorization"] = success.data.theBasicAuthHeaderToUse;
                    this.$state.go('root.app.home');
                }, error => {
                    console.log(error.data.isValidUser);                    
                    this.appWide.isUploadVisible = false;
                    delete this.$http.defaults.headers["common"]["Authorization"];
                    this.TheSweetAlert("Oops...", "Not authorized", "error");                
                });

            }//login
            
        }//Controller
}


An example of Authentication API, AuthApi.ts:

///<reference path="../../typings/angularjs/angular.d.ts"/>
///<reference path="../../shared/dto/LoginDto.ts"/>
///<reference path="../../shared/dto/LoginResponseDto.ts"/>

module InhouseLib {

    export class AuthApi {

        constructor(public $http : ng.IHttpService) {

        }

        verify(username: string, password: string) : ng.IHttpPromise<Dto.LoginResponseDto> {

            var u = new Dto.LoginDto();
            u.username = username;
            u.password = password;


            return this.$http.post<Dto.LoginResponseDto>('/api/member/verify', u);

        }

    }

}


// Angular initialization code:
// mod.service('AuthApi', ['$http', InhouseLib.AuthApi]);



The problem with the caller(Login Controller) of the Authentication API is it does things that should not be of concern to it. One concern that should not be in the controller is the assigning and invalidating of Basic Authentication to angular's $http pipeline.


To improve that. The assigning and invalidating of Basic Authentication information to angular's $http pipeline should be done on AuthApi.ts. Following is an example. First, we must remove the code related to Basic Authentication away from the concerns of Login controller.

login():void {

    this.authApi.verify(this.username, this.password)
        .then(success => {
             console.log(success.isValidUser);
             this.appWide.isUploadVisible = true;
             this.$state.go('root.app.home');
        }, error => {
             console.log(error.isValidUser);                
             this.appWide.isUploadVisible = false;
             this.TheSweetAlert("Oops...", "Not authorized", "error");
        });


}


Then move that concern to AuthApi.ts:

verify(username: string, password: string) : ng.IPromise<Dto.LoginResponseDto> {

    var deferred = this.$q.defer();

    var u = new Dto.LoginDto();
    u.username = username;
    u.password = password;


    this.$http.post<Dto.LoginResponseDto>('/api/member/verify', u)
    .then(success => {

        this.$http.defaults.headers["common"]["Authorization"] = success.data.theBasicAuthHeaderToUse;

        // This goes to then's success callback parameter
        deferred.resolve(success.data);

    }, error => {

        delete this.$http.defaults.headers["common"]["Authorization"];

        // this goes to then's error callback parameter
        deferred.reject(error.data);

    });

    return deferred.promise;

}


If the caller code (Login Controller) has a need receive http status code returned by the server, return a promise of IHttpPromiseCallbackArg with an argument of the other information(Dto.LoginResponseDto) instead.

verify(username: string, password: string) :     
    ng.IPromise<ng.IHttpPromiseCallbackArg<Dto.LoginResponseDto>> 
{

    var deferred = this.$q.defer();

    var u = new Dto.LoginDto();
    u.username = username;
    u.password = password;


    this.$http.post<Dto.LoginResponseDto>('/api/member/verify', u)
        .then(success => {
            this.$http.defaults.headers["common"]["Authorization"] = success.data.theBasicAuthHeaderToUse;

            // This goes to then's success callback parameter
            deferred.resolve(success);

        }, error => {

            delete this.$http.defaults.headers["common"]["Authorization"];

            // this goes to then's error callback parameter
            deferred.reject(error);

        });

    return deferred.promise;

}


To access the other information(Dto.LoginResponseDto), access it via success.data and error.data. For the http status code, access it from success or error parameter, e.g., error.status.

login():void {


     this.authApi.verify(this.username, this.password)
         .then(success => {
             console.log(success.data.isValidUser);
             this.appWide.isUploadVisible = true;
             this.$state.go('root.app.home');
         }, error => {
             console.log(error.data.isValidUser);                
             this.appWide.isUploadVisible = false;
             this.TheSweetAlert(error.status.toString(), "Not authorized", "error");
         });

}

WebStorm Need Tight TypeScript Integration

Seriously.


Whereas Visual Studio Code coding assistance almost works flawlessly. For example Visual Studio Code shows on the tooltip the type of the parameter when hovering over the parameter:



..WebStorm has problem knowing the type:



Wishing there is TypeScriptStorm.

Thursday, September 3, 2015

Fast text search in PostgreSQL

This works in Postgres:


select * from location where location_id = any(array[1,6,8])

However, if you tried to match it against an array on another query, it won't work, even though the another query returns exactly one row only.

select * from location where location_id = any(select location_ids from province p where p.province_name = 'Metro Manila' limit 1)

Output:
ERROR:  operator does not exist: integer = integer[]
LINE 1: select * from location where location_id = any
                                                 ^
HINT:  No operator matches the given name and argument type(s). You might need to add explicit type casts.


To correct it, unnest the array:

select * from location where location_id = any(select unnest(location_ids) from province p where p.province_name = 'Metro Manila')

Might as well use IN:

select * from location where location_id in (select unnest(location_ids) from province p where p.province_name = 'Metro Manila')


There's a drawback on the queries above, it uses equal operator. In Postgres, indexers are bound to operator, and GIN, the indexing technology for indexing array, doesn’t support equal operator. See the detail here: http://stackoverflow.com/questions/4058731/can-postgresql-index-array-columns/29245753#29245753


So to optimize the query, use either of the following:


-- This query works same as the query at the bottom. The query at the bottom is more readable though.
-- The advantage of this query is if the province matches many rows, it would still work, say the province_name uses ILIKE operator.

select  *
from    location l
where   exists
        (
            select  p.location_ids
            from    province p
            where   p.province_name = 'Metro Manila'
                    and array[l.location_id] && p.location_ids
        );

The query uses overlap operator, &&. See the use of supported operators on arrays: http://www.postgresql.org/docs/9.1/static/functions-array.html

-- If province returns exactly one row, this is more readable than above
select   *
from     location l
where    array[l.location_id] && ( select location_ids p from province p where p.province_name = 'Metro Manila' );


Live code: http://sqlfiddle.com/#!15/6ac38/5


Happy Coding!

Monday, August 31, 2015

module.exports has lesser mental model when exporting a module

On my post about sharing TypeScript classes between client-side and server-side. I used exports rather than module.exports. However, as I later learned, it's better to use module.exports as it's the one being directly returned by the require function, module.exports is the real deal. If we uses module.exports rather than exports, we don't need to do this anymore:

class ExternalizedDomain {
    static Person : typeof Domain.Person = require('./shared/Domain/Person').DomainPerson;
    static Country : typeof Domain.Country = require('./shared/Domain/Country').DomainCountry;
}

And also the drawback of the idiom above is it is not compatible with proxyquire:

it("applies interest using stubbed calculator", () => {

 var calculatorStub : any = {};

 var financialCalculator : typeof Domain.FinancialCalculator = 
                  proxyquire('../shared/Domain/FinancialCalculator', { './Calculator': calculatorStub });

 var interestApplied = financialCalculator.applyInterest(200, 0.2);
 expect(interestApplied).toEqual(240);


 calculatorStub.multiply = (a,b) => 6;
 var interestAppliedFromStubbedCalculator = financialCalculator.applyInterest(100, 0.2);
 expect(interestAppliedFromStubbedCalculator).toEqual(6);
});


There's no way in proxyquire to specificy an specific property (e.g., .DomainPerson) of the object assigned to module.exports. To make the external module compatible with proxyquire, do as the following:


class ExternalizedDomain {
    static Person : typeof Domain.Person = require('./shared/Domain/Person');
    static Country : typeof Domain.Country = require('./shared/Domain/Country');
}


Then change the Person.ts and Country.ts to export things on module.exports:

Person.ts:
module Domain {
    export class Country {
        name : string;
    }
}

// Hack for converting internal module to external module
declare var exports: any;
if (typeof exports != 'undefined') {
    module.exports = Domain.Country;
}

Country.ts:
module Domain {
    export class Country {
        name : string;
    }
}


// Hack for converting internal module to external module
declare var exports: any;
if (typeof exports != 'undefined') {
    module.exports = Domain.Country;
}



Definition of Calculator.ts:
///<reference path="../../typings/node/node.d.ts"/>

var isNode = typeof exports !== 'undefined' && this.exports !== exports;


module Domain.Calculator {

    export function multiply(multiplicand: number, multiplier: number): number {

        return multiplicand * multiplier;
    }

    export function divide(dividend: number, divisor: number): number {

        return null;
    }
}



if (isNode) {
    module.exports = Domain.Calculator;
}


Definition of FinancialCalculator.ts:
/// <reference path="../../typings/node/node.d.ts"/>

///<reference path="Calculator.ts"/>


var isNode = typeof exports !== 'undefined' && this.exports !== exports;

var calculator : typeof Domain.Calculator = isNode ? require('./Calculator') : Domain.Calculator;


module Domain.FinancialCalculator {

    export function applyInterest(amount: number, percentInterest: number): number {

        return calculator.multiply(amount, 1 + percentInterest);
    }

}


if (isNode) {
    module.exports = Domain.FinancialCalculator;
}


On next post, I'll show the difference between TypeScript's classes and module when it comes to proxyquire.


Happy Coding!