Saturday, March 30, 2019

Paper Rock Scissors Game Coding Challenge

Alright I'm trying out this well known coding challenge.

Paper-Rock-Scissors is a hand game usually played by two people, where players simultaneously form one of three shapes with an outstretched hand.

  • The rock beats scissors by blunting it
  • The scissors beat paper by cutting it
  • The paper beats rock by wrapping it

If both players throw the same shape, it is a draw.

My Solution

I'm doing it in Java this time. I try to keep it simple and create the solution as a console application.

Player

Computer and Human for now, both implementing Player interface.

public interface Player {

    String getName();

    Move getMove();
}

Computer player takes in a Random object which simply going to pick a random Move whenever asked.

public class Computer implements Player {

    private final String name;
    private final Random random;

    public Computer(String name, Random random) {
        this.name = name;
        this.random = random;
    }

    @Override
    public String getName() {
        return name;
    }

    @Override
    public Move getMove() {
        MoveSelection[] moveSelections = MoveSelection.values();
        return moveSelections[random.nextInt(moveSelections.length)];
    }
}

Ignore the MoveSelection[] for now, it will become clear when we get to the win / lose / draw logic below.

Human player just has a setter to set Move and a getter to retrieve it back.

public class Human implements Player {

    private final String name;
    private Move move;

    public Human(String name) {
        this.name = name;
    }

    public void setMove(Move move) {
        this.move = move;
    }

    @Override
    public String getName() {
        return name;
    }

    @Override
    public Move getMove() {
        return move;
    }
}

I created two ways to feed human player input into the game:

  • CsvFileInputParser: reads only valid (case insensitive) user input ie. P, R, S, PAPER, ROCK, SCISSORS specified in a csv file and ignores the invalid ones.
  • Interactive from command line: read on for more details

Gamemaster

The purpose of having a gamemaster class is to avoid having too much code in the console application main which is rather difficult to test. Although we can't really fully unit test interactive console application, we still can get some test coverage by doing this.

For some reason, I ended up with three different kinds of gamemaster:

  • HumanVsComputerGamemaster: at first I started with computer vs a very dumb human player whose moves are read in from a csv file. This is what this gamemaster is doing.
  • ComputerVsComputerGamemaster: once I got the simplest working, I started thinking since Player is an interface, we can actually mix and match the players, we can even have a computer player pitches against another computer player. This is how this gamemaster came to be. In fact, it is actually easier to implement than the human vs computer version, so it doesn't take a lot of time to create this.
  • InteractiveHumanVsComputerGamemaster: the interactive gamemaster is built in a similar way as the other two gamemasters. It is imho the hardest to implement as I've never had to read from console using Java, so I had to google it a bit and I decided to use Scanner for this. Note that I'm not sure how to unit test this interactive mode, but I have enough unit tests for the other two gamemasters to be rather confident that it is working as they're all using the same base class ConsoleTwoPlayerGamemaster.

Win / Lose / Draw Logic

The win / lose logic is centralised in MoveSelection enum that implements Move interface

public interface Move {

    List<Move> winsAgainst();

    List<Move> losesTo();
}

I purposely make winsAgainst() and losesTo() a list to sort of model a matrix-like decision making. This might make things a bit complex but I think it's worth it for the extension possibility. So for example if we introduce a new move BLUNT_SCISSORS, that loses to PAPER, ROCK and SCISSORS and wins against nothing, then all we need to do is:

  • add BLUNT_SCISSORS to PAPER, ROCK, SCISSORS's winsAgainst() list
  • add PAPER, ROCK, SCISSORS to BLUNT_SCISSORS's losesTo() list
  • leave BLUNT_SCISSORS's winsAgainst() list empty
and we're pretty much done.

Suppose we introduce yet another move CRUMBLING_ROCK and let's say it loses to ROCK, SCISSORS and BLUNT_SCISSORS (all crumbles the weak rock) and it wins against PAPER (the paper cannot contain all the crumbles), then all we need to do is:

  • add CRUMBLING_ROCK to ROCK, SCISSORS and BLUNT_SCISSORS's winsAgainst() list
  • add CRUMBLING_ROCK to PAPER's losesTo() list
  • add PAPER to CRUMBLING_ROCK's winsAgainst() list

I understand that it probably does not make sense for any player to choose a move with only a few items in the winsAgainst() list as it reduces the chance of winning, however it might make sense if we have a new computer player type for example an easy level computer player.

I also understand that there's a room for win/lose logic inconsistency here everytime a new move is introduced as I have not implemented a validation that every Move should appear in every other Move's winsAgainst() OR losesTo() list, but never in both list. Using the examples I mentioned above, a correct logic should look like this:

public enum MoveSelection implements Move {

    PAPER {

        @Override
        public List<Move> winsAgainst() {
            return List.of(ROCK, BLUNT_SCISSORS);
        }

        @Override
        public List<Move> losesTo() {
            return List.of(SCISSORS, CRUMBLING_ROCK);
        }
    },

    ROCK {

        @Override
        public List<Move> winsAgainst() {
            return List.of(SCISSORS, BLUNT_SCISSORS, CRUMBLING_ROCK);
        }

        @Override
        public List<Move> losesTo() {
            return List.of(PAPER);
        }
    },

    SCISSORS {

        @Override
        public List<Move> winsAgainst() {
            return List.of(PAPER, BLUNT_SCISSORS, CRUMBLING_ROCK);
        }

        @Override
        public List<Move> losesTo() {
            return List.of(ROCK);
        }
    },

    BLUNT_SCISSORS {

        @Override
        public List<Move> winsAgainst() {
            return List.of(CRUMBLING_ROCK);
        }

        @Override
        public List<Move> losesTo() {
            return List.of(PAPER, ROCK, SCISSORS);
        }
    },

    CRUMBLING_ROCK {

        @Override
        public List<Move> winsAgainst() {
            return List.of(PAPER);
        }

        @Override
        public List<Move> losesTo() {
            return List.of(ROCK, SCISSORS, BLUNT_SCISSORS);
        }
    }
}

Here we can see for example that CRUMBLING_ROCK appears in either winsAgainst() OR losesTo() list of each of the other enums, but never on both lists.

TwoPlayerJudge

The TwoPlayerJudge as its name says, takes in two Player's and will determine the winner from the first player's point of view. The TwoPlayerJudge uses the above decision matrix to decide whether the first player is winning, losing or it's a draw. The TwoPlayerJudge has some simple input validation, but as mentioned above, I have not implemented validation on the "matrix". Furthermore, if I do have time to implement this, I probably won't make it the responsibility of the judge.

public TwoPlayerResult judgeFromFirstPlayerPointOfView(Player firstPlayer, Player secondPlayer) {
    Move firstPlayerMove = firstPlayer.getMove();
    validateMove(firstPlayerMove);

    Move secondPlayerMove = secondPlayer.getMove();
    validateMove(secondPlayerMove);

    MoveResult moveResult = MoveResult.DRAW;
    if (firstPlayerMove.losesTo().isEmpty() || firstPlayerMove.winsAgainst().contains(secondPlayerMove)) {
        moveResult = MoveResult.WIN;
    } else if (firstPlayerMove.winsAgainst().isEmpty() || firstPlayerMove.losesTo().contains(secondPlayerMove)) {
        moveResult = MoveResult.LOSE;
    }

    return new TwoPlayerResult(firstPlayerMove, secondPlayerMove, moveResult);
}

public void validateMove(Move move) {
    if (move == null) {
       throw new IllegalStateException("Cannot judge as player does not have move set");
    }

    if (move.winsAgainst().isEmpty() && move.losesTo().isEmpty()) {
       throw new IllegalStateException("Cannot judge as player cannot both wins against nothing and loses to nothing");
    }

    if (move.winsAgainst().equals(move.losesTo())) {
       throw new IllegalStateException("Cannot judge as player cannot both wins against and loses to the same move");
    }
}

TwoPlayerResult is just a container class to store the first and second player move and the move result (WIN, LOSE or DRAW).

public TwoPlayerResult(Move firstPlayerMove, Move secondPlayerMove, MoveResult moveResult) {
    this.firstPlayerMove = firstPlayerMove;
    this.secondPlayerMove = secondPlayerMove;
    this.moveResult = moveResult;
}

Main

This is the entry point to run the console application.

public class Main {

    public static void main(String[] args) {

        Human human = new Human("Hooman");
        Random random = new Random();
        Computer computer = new Computer("Robo", random);
        TwoPlayerJudge judge = new TwoPlayerJudge();

        // The 3 kinds of gamemaster. Comment out the ones you don't want to run.
        // 1) Human vs Computer: human input is read from a csv file under resources
        runHumanVsComputer(human, computer, judge);

        // 2) Computer vs Computer: enter number of rounds and let them fight each other
        runComputerVsComputer(random, computer, judge, 10);

        // 3) Interactive Human vs Computer: human input is read from the console
        runInteractiveHumanVsComputer(human, computer, judge);
    }

    private static void runHumanVsComputer(Human human, Computer computer, TwoPlayerJudge judge) {

        String filePath = Objects.requireNonNull(Main.class.getClassLoader().getResource("HumanMoves.csv")).getFile();
        CsvFileInputParser csvFileParser = new CsvFileInputParser(filePath);
        ConsoleTwoPlayerGamemaster gamemaster = new HumanVsComputerGamemaster(human, csvFileParser.readMoves(), computer, judge);
        System.out.println(gamemaster.startGame());
    }

    private static void runComputerVsComputer(Random random, Computer computer1, TwoPlayerJudge judge, int numberOfRounds) {

        Computer computer2 = new Computer("Robo Wannabe", random);
        ConsoleTwoPlayerGamemaster gamemaster = new ComputerVsComputerGamemaster(computer1, computer2, judge, numberOfRounds);
        System.out.println(gamemaster.startGame());
    }

    private static void runInteractiveHumanVsComputer(Human human, Computer computer, TwoPlayerJudge judge) {

        ConsoleTwoPlayerGamemaster gamemaster = new InteractiveHumanVsComputerGamemaster(human, computer, judge);
        gamemaster.startGame();
    }
}

Source Code

https://github.com/velianarie/PaperRockScissorsGame

Wednesday, October 31, 2018

Feature Toggling Graphql Service with LaunchDarkly

For a project I was working on, we have a graphql service that retrieves clients positions from a SQL database. Under the hood there are some complex SQL stored procedures joining multiple SQL tables and complex data processing before they are finally in a usable shape. We want to move all this complexity to a Hive Big Data project which is more suitable to handle this type of data massaging and have our project consumed a Hive REST API.

The plan is to make sure that once the Hive REST API is live or even during UAT, the consumers of our graphql service do not notice any difference. And if something goes wrong with Hive, we can switch back to good old SQL with very minimal code change. This is where I think LaunchDarkly comes in handy as a fallback scenario. Fortunately I have some prior experience working with it.


The Toy Example

Now to illustrate what I'm dealing with, I have created a toy graphql service. Suppose we have a graphql query that retrieve clients positions defined as:

   type Query {
      positions: [String]
   }

Suppose the result of the query sourced from SQL is:

   {
     "data": {
       "positions": [
         "Sql1",
         "Sql2",
         "Sql3"
       ]
     }
   }

We want to be able to switch to Hive on the fly so that the same query returns:

   {
     "data": {
       "positions": [
         "Hive1",
         "Hive2",
         "Hive3"
       ]
     }
   }

Without feature toggling management, one way to achieve this is to introduce a boolean flag ie. use Hive vs use SQL. The graphql query definition becomes something like this:

   type Query {
      positions(useHive:Boolean!): [String]
   }

This solution isn't always possible because this means we have to change our public API. With LaunchDarkly there's no need to alter our public API, all the code needed for feature toggling happens internally.

Source Code

Using LaunchDarkly

If you're new to LaunchDarkly, see my other post for some background information how to get started.

Create a new feature flag and call it 'position-hive'. When this feature is switched ON, we'll make the positions query returning data from Hive. We can also target a specific set of users who are allowed to use this feature and another set who aren't.

Since I'm using node.js to build this graphql service, we will need LaunchDarkly SDK for Node.js. Install the Node.js SDK with npm:

    npm install ldclient-node --save

We want to make sure that LaunchDarkly client and user are initialised only once when the graphql service started. Once they are initialised, we passed them into the graphql context so we can retrieve these back when needed. Now in the real world there's usually user authentication and authorisation before we can use a service. I'm not going to cover that here, I'm just assuming that's already happening automagically and we can get the user id and name from the graphql request argument.

According to LaunchDarkly SDK documentation, the LaunchDarkly client will emit a 'ready' event when it has been initialized and can serve feature flags. waitForInitialization() function is an alternative to this 'ready' event. If your application uses promises to manage asynchronous operations, which is the case here, this is the recommended way to interact with the SDK. When LaunchDarkly client is ready and can serve feature flags then retrieving a flag setting for a given user can be done this way:

   ldClient.variation(FEATURE_KEY, ldUser, FEATURE_DEFAULT)
     .then(function(featureValue) {
         // application code
     });
The FEATURE_KEY is 'position-hive' and FEATURE_DEFAULT is either true or false. I set it to false in our case as I want our graphql service to use SQL data by default.

Source Code

https://github.com/velianarie/graphql-launchdarkly

Monday, October 29, 2018

Wi-Fi Problem Ubuntu 18.04 USB Boot on MacBook Pro

Always wanted to try Ubuntu but didn't want to risk changing anything in your machine? Well the community has thought about this. They termed it as live run, basically having the whole OS running from a CD, DVD, USB or any portable device you could think of, no installation needed. There are quite a few tutorials explaining how to do this, so I won't bother repeating it here. This tutorial is a good one to get it on any USB stick. I chose USB because that's what I had near me.

Booting from USB works like a charm, everything seems to work except the Wi-Fi! What can a machine do without internet these days? I looked into the Wi-Fi Settings and it's telling me it couldn't find a Wi-Fi adapter.

Unfortunately I don't have an extra machine to google for solution, so I had to use the good old Ethernet cable to connect to internet and that fortunately worked, I don't know what I would do if I had one of those thin laptops. Anyway if you google Ubuntu Wi-Fi problem, it will get you hundreds of hits... seems like this is a well-known problem and my research narrowed it down to missing driver.

To know what you're missing, the first thing you need to do is identify the network card you have. The command in Linux is lspci -vvnn | grep -A 9 Network.

   ubuntu@ubuntu:~$ lspci -vvnn | grep -A 9 Network
   02:00.0 Network controller [0280]: Broadcom Limited BCM4331 802.11a/b/g/n [14e4:4331] (rev 02)
      Subsystem: Apple Inc. AirPort Extreme [106b:00f5]
      ...
      Latency: 0, Cache Line Size: 256 bytes
      Interrupt: pin A routed to IRQ 17
      Region 0: Memory at a0600000 (64-bit, non-prefetchable) [size=16K]
      Capabilities: 
      Kernel driver in use: bcma-pci-bridge
      Kernel modules: bcma
This means I have:
  • The Chip ID: BCM4331
  • The PCI-ID: 14e4:4331
  • Kernel driver in use: bcma-pci-bridge

Fortunately I can see I don't have wl driver, this is easily remedied by installing bcmwl-kernel-source

   sudo apt install bcmwl-kernel-source
My Wi-Fi works afterwards.

If you're not as lucky as I am, knowing your network card is a first step to google for some more solution.

My MacBook Pro spec: macOS High Sierra, 2.9 Ghz Intel Core i7, 8 GB 1600 MHz DDR3

Saturday, July 14, 2018

Pluto Rover Coding Challenge

This is a spin-off of the famous Mars Rover coding assignment.

The Assignment

After NASA’s New Horizon successfully flew past Pluto, they now plan to land a Pluto Rover to further investigate the surface. You are responsible for developing an API that will allow the Rover to move around the planet. As you won’t get a chance to fix your code once it is on board, you are expected to use test driven development.

To simplify navigation, the planet has been divided up into a grid. The rover's position and location is represented by a combination of x and y coordinates and a letter representing one of the four cardinal compass points. An example position might be 0, 0, N, which means the rover is in the bottom left corner and facing North. Assume that the square directly North from (x, y) is (x, y+1).

In order to control a rover, NASA sends a simple string of letters. The only commands you can give the rover are ‘F’,’B’,’L’ and ‘R’

  • Implement commands that move the rover forward/backward (‘F’,’B’). The rover may only move forward/backward by one grid point, and must maintain the same heading.
  • Implement commands that turn the rover left/right (‘L’,’R’). These commands make the rover spin 90 degrees left or right respectively, without moving from its current spot.
  • Implement wrapping from one edge of the grid to another. (Pluto is a sphere after all)
  • Implement obstacle detection before each move to a new square. If a given sequence of commands encounters an obstacle, the rover moves up to the last possible point and reports the obstacle.

Here's an example

  • Let's say that the rover is located at 0,0 facing North on a 100x100 grid.
  • Given the command "FFRFF" would put the rover at 2,2 facing East.

Tips!

  • Don't worry about the structure of the rover. Let the structure evolve as you add more tests.
  • Start simple. For instance you might start with a test that if at 0,0,N with command F, the robots position should now be 0,1,N.
  • Don’t worry about bounds checking until step 3 (implementing wrapping).
  • Don't start up/use the debugger, use your tests to implement the kata. If you find that you run into issues, use your tests to assert on the inner workings of the rover (as opposed to starting the debugger).

My Solution

I started with defining a few things:

  • Enum Command that represents each command (forward, backward, left and right)
  • Enum Orientation that represents the four cardinal directions (north, east, south and west)
  • Class Position that represents the current rover's position in the grid
public enum Command 
{
   Forward,
   Backward,
   Left,
   Right
}
  
public enum Orientation
{
   North,
   South,
   East,
   West
}
    
public class Position
{
   private readonly int x;
   private readonly int y;
   private readonly Orientation orientation;
    
   public Position(int x, int y, Orientation orientation) 
   {
      this.x = x;
      this.y = y;
      this.orientation = orientation;
   }
}

Then I define a static class InputParser that knows how to parse a character from the string of input letters eg. "FFRFF" into a Command. For simplicity, I won't show the code here, it's just a bunch of switch case statements.

Next I define a class Pluto which represents the surface/grid where the rover can move around on. I also think it makes sense to have this class responsible for holding information of the obstacles.

public class Pluto
{
   private readonly int width;
   private readonly int length;
   private readonly List<Tuple<int, int>> obstacles;
     
   public Pluto(int width, int length) 
   {
      this.width = width;
      this.length = length;
      obstacles = new List<Tuple<int, int>>();
   }

   public int Width 
   {
      get { return width; }
   }
   
   public int Length
   {
      get { return length; }
   }
     
   public List<Tuple<int, int>> Obstacles 
   {
      get { return obstacles; }
   }

   public void AddObstacle(Tuple<int, int> obstacle) 
   {
      if (obstacles.Contains(obstacle)) 
      {
         obstacles.Remove(obstacle);
      }
      
      obstacles.Add(obstacle);
   }
}

Finally I define a Rover class that can be initiated as so:

public Rover(int x, int y, Orientation orientation) 
{
   this.x = x;
   this.y = y;
   this.orientation = orientation;
}

This class holds a reference to Pluto with the help of a DeployTo function:

public void DeployTo(Pluto pluto)
{
   this.pluto = pluto;
}

and implements each single move logic as so:

  1. Determine the candidate new position
  2. int candidateX = x;
    int candidateY = y;
    switch (command)
    {
       case Command.Forward:
          if (orientation == Orientation.North) candidateY = candidateY + 1;
          else if (orientation == Orientation.South) candidateY = candidateY - 1;
          else if (orientation == Orientation.East) candidateX = candidateX + 1;
          else candidateX = candidateX - 1;
          break;
       case Command.Backward:
          if (orientation == Orientation.North) candidateY = candidateY - 1;
          else if (orientation == Orientation.South) candidateY = candidateY + 1;
          else if (orientation == Orientation.East) candidateX = candidateX - 1;
          else candidateX = candidateX + 1;
          break;
       case Command.Left:
          if (orientation == Orientation.North) orientation = Orientation.West;
          else if (orientation == Orientation.South) orientation = Orientation.East;
          else if (orientation == Orientation.East) orientation = Orientation.North;
          else orientation = Orientation.South;
          break;
       case Command.Right:
          if (orientation == Orientation.North) orientation = Orientation.East;
          else if (orientation == Orientation.South) orientation = Orientation.West;
          else if (orientation == Orientation.East) orientation = Orientation.South;
          else orientation = Orientation.North;
          break;
       default:
          throw new Exception($"Command '{command}' is not valid.");
    }
  3. Ask Pluto for positions of the obstacles
  4. If there is an obstacle on this candidateX and candidateY position, the Rover just stays on its current x and y position, although it still can change its orientation. If there is no obstacle, candidateX and candidateY is the new position.
  5. If candidateX or candidateY position exceeds Pluto's width and length, adjust candidateX and candidateY accordingly by deducting the width and length.

Things to Improve

If I have more time, here are a few things I want to improve:

  • Replace Tuple<int,int> with Coordinate class
  • Replace Rover(int x, int y, ...) with this Coordinate class
  • if else statements in Move() strikes me as a bit of a code smell (see below my thoughts of how to possibly improve this)
  • Move() function is too long, there's code smell here. Moving logic is tangled with obstacles detection logic. They need to be separated. Think of a change of requirement for example if the Rover is deployed to a planet that's not a wrapped grid, but a torus or an infinite grid.
  • Abstract out Pluto to guard for change of requirement as mentioned before. Maybe call it IPlanet?

My thoughts to improve Command Left and Right

We could use some mathematical tricks like assigning integer number to the Orientation enum. Pick an orientation, assign integer value 1 then go clockwise and assign the next integer value, so something like: East = 1, South = 2, West = 3, North = 4.

Apply the following for Left:

  if (current orientation - 1) < 1, then get the previous orientation as if the enum is a circle 

  else (current orientation - 1) 
  

Example:

  • current orientation: N ==> (4 - 1) = 3 (West)
  • current orientation: S ==> (2 - 1) = 1 (East)
  • current orientation: E ==> (1 - 1) = 0 (the previous orientation is North)
  • current orientation: W ==> (3 - 1) = 2 (South)

Apply similar trick to Right, it is the opposite of Left after all:

  if (current orientation + 1) > 4, then get the next orientation as if the enum is a circle 

  else (current orientation + 1) 
  

Example:

  • current orientation: N ==> (4 + 1) = 5 (the next orientation is East)
  • current orientation: S ==> (2 + 1) = 3 (West)
  • current orientation: E ==> (1 + 1) = 2 (South)
  • current orientation: W ==> (3 + 1) = 4 (North)

This trick would reduce the 8 if else statements to 2 functions.

Note however that I haven't thought about whether the two above pseudocodes would work for different enumeration value ie. if you start with North = 1 for example.

For Command Forward and Backward, we can see the following pattern:

  • Forward North = Backward South
  • Foward South = Backward North
  • Forward East = Backward West
  • Forward West = Backward East

So perhaps we can reduce some of the if else statements based on these facts.

Source Code

https://github.com/velianarie/PlutoRover

Saturday, March 31, 2018

JavaScript Spread Operator and Destructuring

I just found out about this amazing (now rather old) feature of ES6 while googling of ways to clone an object then excluding some of the properties. We basically can do this by combining spread operator with destructuring.

So supposed we have this recipe object and we want to turn this recipe into an alcohol free recipe ie. taking the rum and kirsch out of the recipe.

Output:

   Original recipe: {"egg":45,"butter":60,"sugar":150,"rum":10,"kirsch":10,"milk":120,"flour":150}
   Alcohol free recipe: {"egg":45,"butter":60,"sugar":150,"milk":120,"flour":150}

Saturday, February 3, 2018

Friday, November 17, 2017

[Learning Notes]: TDD for React/Redux in an Isomorphic Application by Hany Elemary

I felt lucky I managed to watch this video before our Safari Books Online subscriptions were discontinued.

History

Hany Elemary begins with a little history of how isomorphic applications came to be. They evolve roughly as follows:
  1. Full-page refresh
    • Thin client (DOM manipulation, form validation, animations)
    • Rich server (routing, views, application logics, storage)
  2. Rich Internet application with AJAX request
    • Rich client (DOM manipulation, form validation, animations + views, application/display logic)
    • Thin server (initial bootstrap, API, storage)
  3. Single Page Application Isomorphic
    • Client (DOM manipulation, form validation, animations)
    • Shared code (routing, views, application/display logic) => runs client-side or server-side based on the request's context
    • Server (API, storage)
To make it easier for beginners to understand the differences between these developments, Hany mentions an analogy of a server in a restaurant and what they do when a customer orders a dish:
  1. Full-page refresh
    • First order: go buy pots pans, groceries, come back, cook & serve
    • Subsequent orders: go buy pots pans, groceries, come back, cook & serve
  2. Rich internet application with AJAX request
    • First order: go buy pots pans, come back, go get groceries, come back, cook & serve
    • Subsequent orders: go get groceries, come back, cook & serve
  3. Single Page Application Isomorphic
    • First order: go buy pots pans, get groceries, come back, cook & serve
    • Subsequent orders: go get groceries, come back, cook & serve

Sequence diagram of isomorphic application

Initial request
  1. User enters URL in browser
  2. Browser requests data from server
  3. Server fetches data from API/Database
  4. API/Database returns data to server eg. JSON
  5. Server generates & serves markup back to browser
  6. Browser render page
Subsequent requests
  1. User navigates in browser
  2. Browser fetches data straight from API/Database
  3. API/Database returns data to browser
  4. Browser renders page

React/Redux Overview

Paradigm shift in front-end development

  • React is library NOT framework
  • React only handles view layer eg. no AJAX, no Promises, no event system, etc. So why do we need it? Well sometimes we only need small composable library rather than a big ecosystem where you only use a tiny fraction of it. Get rid of libraries you don't need and get new libraries when you need it
  • React is component-based architecture: reusable components
  • Web developers often told not to mix HTML with JavaScript because HTML cannot be minified and it also cluttered your components. React uses HTML within JavaScript (JSX). JSX is pre-processor step that compiles your HTML into JavaScript so it could be minified.
  • Virtual DOM and diffing algorithm. Virtual DOM is in-memory representation of the real DOM, it has no knowlege to browser or environment it's run in, it's essentially a data structure. Diffing algorithm makes sure only the difference between virtual and real DOM is committed to the real DOM.

Deterministic view rendering

We know exactly which view is rendered first. In asynchronous function we won't know, whichever request got a response first, it'll be rendered first (race condition) ie. it's non-deterministic.

Event Sourcing

Event sourcing is technique that cares about sequences of events that lead to a specific outcome rather than the outcome itself. We want to store all events that occurred through our system as opposed to the outcomes that have happened. Characteristic: no update, no delete, it's an append-only log of events. In accounting terms: general ledger. Your book of records becomes stream of events, your source of truth. Advantages:
  • We have audit trail of all events that have happened in our system
  • You can replay a sequence of events to bring the application to a very specific state

Redux was inspired by Event Sourcing

  • Single source of truth: Redux Store
  • State is read-only/immutable. We only appending things, we're not updating or deleting things
  • Changes (ie. actions not events!) are made with pure functions (reducers). Reducers: given the same parameters will always output the same result, no side effect, no network operation, etc

Developer's workflow

  1. Write action / action creator
  2. Write component reducer to return the new state
  3. Determine the shape of your state / data - root reducer
  4. Write component that dispatches your action
  5. Connect component to store to subscribe to changes

TDD with React/Redux - Simple Components

Test pyramid

  1. UI
  2. Driven through the browser. Ensure the app is healthy in very high level, only test business critical features that represent overall workflows. Tend to be very slow and very expensive to maintain.
  3. Service (service level test, contract test)
  4. Expecting specific response for API given specific request. If API changes, test should fail. Protecting your application from changes in downstream dependencies or if your application is dependent on an API that you don't own.
  5. Unit
  6. Responsible for example display logic, behaviour, integration between components. Should be very fast and inexpensive to maintain.

Testing framework

  • Mocha - Test framework: assertions, expectations
  • Sinonjs - Stubbing framework: fake and spy on function calls
  • Mochawesome - Reporting: HTML reports that generate with each run, important to integrate with CI/CD pipeline
  • Enzyme - React test utility: rendering mechanism, access to components instances and children
  • Istanbul - Reporting: code coverage

TDD with React/Redux - Async Operation

Redux Saga

  • A library that aims to make side-effects easier and better.
  • What is side effects? network / asynchronous operations like fetching data. We can't guarantee that the same request will always be successful.
  • Reads like synchronous code. Think of Redux Saga as separate thread in your application.
  • ES6 generators makes an asynchronous code appears synchronous.
  • Redux Saga is a middleware. (PS: middleware is some code you can put between the framework receiving a request and the framework generating a response eg. logging)

Testing frameworks

  • Nock - HTTP mocking: useful for testing components integration with APIs, E2E tests
  • Mountebank - HTTP stubbing over the wire: useful for testing components integration with APIs, E2E tests, simulating failure modes.
  • Mountebank allows you to create an actual server and then mock the HTTP request and response. You can do pattern matching ie. if I see this pattern in the request then return this particular response. Another feature is latency ie. this request will return this response after n seconds. This can be used to simulate loading in your application.

Thursday, August 10, 2017

Getting VSTO project to Jenkins

I was a bit frustrated last week with a task that seemed so simple but took me the whole week to get it done.. and in a rather dirty way I must say. We have a legacy VSTO project that we want to get into Jenkins. This project has a legacy installer that's built as a .vdproj (Visual Studio Installer project). This has always been built locally on a developer machine and released from there. I know, I know, that's nasty but it is what it is. So I was trying to do two things: getting the solution AND the installer to build on Jenkins.

Problems Encountered

Vdproj has been deprecated

Microsoft only supports vdproj that shipped with VS2010. We don't have license for VS2010 anymore. We're using VS2015. Fortunately there's an extension we can install for higher VS version. Download link here.

MSBuild does not know anything about .vdproj file

This is very problematic because MSBuild does not know how to build .vdproj, only Visual Studio does, so we have to install Visual Studio in Jenkins. The command to automate this is:

devenv /SolutionName.sln /project InstallerName.vdproj /build Release
Make sure that you call this from Visual Studio Developer Command Prompt.

There is however a pre-build validation that the whole projects in the solution should load. The VSTO project cannot load without having Office installed. So we have to install Office.
You will see the following error if you don't:

ERROR: Cannot find outputs of project output group '(unable to determine name)'. 
Either the group, its configuration, or its project may have been removed from the solution.

Cryptic .vdproj pre-build validation error

ERROR: An error occurred while validating. HRESULT = '8000000A'
Googling leads me to this page. Yes unfortunately it is a registry hack. Create a DWORD key with the name "EnableOutOfProcBuild" and set it's value to zero in
HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\14.0_Config\MSBuild\

Why not use ClickOnce as installer?

I did try to create an installer using ClickOnce, but came across another problem with a different computed hash in the manifest. This is most likely caused by an xml transformation specific to this application that happens after the hash has been computed.

Conclusion

Avoid VSTO at all cost. I have to pollute the machine where Jenkins is run on to make this legacy application works. Here's a list:
  • Install Visual Studio 2015
  • Install Microsoft Visual Studio 2015 Installer Projects extension
  • Install Microsoft Office
  • Install Visual Studio 2010 Tools for Office Runtime (provides the COM 'glue' between Excel and C#)
  • Hack the registry

Wednesday, May 31, 2017

Feature Toggling with LaunchDarkly

Feature toggle is a software development technique allowing developers to modify system behaviour during run time without code change[1]. Feature toggle is used to enable or disable features. We can for example enable some features for testing even before they are ready for release, but disabled these same features in production.

LaunchDarkly is a feature flag rather than a feature toggle[2]. A toggle implies a two-state condition while a flag can return multiple states or values. In this blog I’m just going to show you how to use LaunchDarkly as a feature toggle to keep things simple.

Project environments

By default LaunchDarkly provides 2 environments, Production and Test. This can be managed in Account settings.

Feature flags

Create a new feature flag by entering:

  • Name
  • Key
  • Description (optional)
  • Flag type

Note that Key is generated automatically as we type in Name. We can alter this if we’re not happy with the auto generated Key. Before saving the flag, please make sure you’re happy with the Key representation. As far as I’m aware, there’s no way we can edit the Key once the flag is created. We will need to delete the whole flag and create a new one with the desirable Key representation. The Key is important here as it is going to be used in your application.

Using LaunchDarkly in your C# application

  1. First thing you need to do is install LaunchDarkly SDK using NuGet in the appropriate project of your C# solution.
  2. Install-Package LaunchDarkly.Client
  3. Import the LaunchDarkly package to your class.
  4. using LaunchDarkly.Client;
  5. Create a new LdClient with your environment-specific SDK key. I'm using Production SDK key here.
  6. var client = new LdClient("sdk-123a4bcd-5ef6-78g0-hi12-34j56k7890l1");
  7. Create a user. I'm only interested in the user name and machine name. You can adjust this according to your need.
  8. var user = User.WithKey(System.Environment.UserName)
                   .AndSecondaryKey(System.Environment.MachineName);
  9. Retrieve LaunchDarkly toggle value.
  10. var isEnabled = client.BoolVariation("calc-scenario-mappings-crud-buttons", user);
    isEnabled returns true if the feature flag is switched ON and returns false if the feature flag is switched OFF.
  11. Use this isEnabled value where you want to enable or disable features in your application.
  12. I'm using LaunchDarkly in a WPF application called CALC to disable the CRUD buttons in Scenario Mappings screen. I already have a view model with a logic that determines whether the CRUD buttons need to be enabled or disabled, so all I need to do is just append this boolean value to the existing logic.

Dev console and run result

Dev console is a handy event listener, it’ll show events as they stream in, live. Have this console open before you run your application.

I’ll run CALC with the feature flag first switched OFF and then switched ON so we can see the different effects.

On the left tab, we see that when the flag is switched OFF, the CRUD buttons are all disabled. On the right tab, when the flag is switched ON, the CRUD buttons are enabled.

Note that the reason why the Dev console show 2 events is because we have 2 tab items (Index and Curve) in the Scenario Mappings screen.

Monday, April 24, 2017

Thinking of using GetCallingAssembly()? Think again

When trying to run a RESTful WCF service in our UAT environment, we had a runtime error saying that an embedded resource could not be loaded. We didn't have this problem locally and also not in DEV environment. The difference between these environments is the build mode. UAT is built in Release, DEV is built in Debug. Running it in Debug mode worked fine, the exception seemed to only happen in Release mode. What made it worse was when we built a Release version and tried to debug that version, it worked fine. Long story short, we managed to narrow it down to the optimized code in Release mode ('Optimize code' check box in Visual Studio) and the use of GetCallingAssembly() in the assembly that has the embedded resource.



So what does 'Optimize code' do and why doesn't it work well with GetCallingAssembly()?

When code optimization is enabled, the JIT compiler moves codes around, for example it dynamically inlines some functions to eliminate the overhead of function calls speed. This means in runtime the optimization might change the code created by the JIT compiler, consequently returning a different assembly than what we had in mind when we decided to use GetCallingAssembly(). This is explained in an old MSDN documentation's remarks:

If the method that calls the GetCallingAssembly() method is expanded inline by the compiler (that is, if the compiler inserts the function body into the emitted Microsoft intermediate language (MSIL), rather than emitting a function call), then the assembly returned by the GetCallingAssembly() method is the assembly containing the inline code. This might be different from the assembly that contains the original method. To ensure that a method that calls the GetCallingAssembly() method is not inlined by the compiler, you can apply the MethodImplAttribute attribute with MethodImplOptions.NoInlining.


Code inlining

What is code inlining? According to wikipedia:

In computing, inline expansion, or inlining, is a manual or compiler optimization that replaces a function call site with the body of the called function

Let's illustrate this with some codes. The illustration is taken from MSDN documentation.

namespace Assembly1
{
    public Assembly Method1()
    {
        return Assembly.GetCallingAssembly();
    }
}

namespace Assembly2
{
    public Assembly Method2()
    {
        return Method1();
    }
}

namespace Assembly3
{
    public Assembly Method3()
    {
        return Method2();
    }
}

When Method1() is not inlined, GetCallingAssembly() returns Assembly2.
When Method2() is not inlined, GetCallingAssembly() returns Assembly2.

When Method1() is inlined:

namespace Assembly3
{
    public Assembly Method3()
    {
        return Method2();
    }
}

namespace Assembly2
{
    public Assembly Method2()
    {
        return Assembly.GetCallingAssembly(); // Method1() is inlined
    }
}
GetCallingAssembly() returns Assembly3.

When Method2() is inlined:

namespace Assembly3
{
    public Assembly Method3()
    {
        return Method1();  // Method2() is inlined
    }
}

namespace Assembly1
{
    public Assembly Method1()
    {
        return Assembly.GetCallingAssembly();
    }
}
GetCallingAssembly() returns Assembly3.

Conclusion

Next time you're thinking about using GetCallingAssembly(), think again. Think of how it's going to be used outside of your project. Is anyone going to enable code optimization? Because if this is the case, GetCallingAssembly() might return unexpected assembly. There are two ways to go about this:

  1. Make sure to apply MethodImplAttribute attribute with MethodImplOptions.NoInlining
  2. Use GetExecutingAssembly() instead because it's not prone to JIT inlining


Further readings

Wednesday, February 1, 2017

JavaScript First Impressions - From a C# Developer POV

The company I work for has recently decided to do web development and as someone who has mostly done desktop development and not much experience in any web development, I'm quite excited about it. I decided to start with the basic, JavaScript. I'll list some of the things I like or find interesting and also things I don't quite like or seem counter-intuitive about JavaScript. I'll put a disclaimer here that this is all my personal opinion and first impressions as a C# developer who's never had any commercial exposure to JavaScript and I might be seriously biased towards C# and unconsciously comparing it with C# with a love-hate feeling and quite possibly more hate in this case.

Like

Syntax-wise similar to C#

C# JavaScript
Variable declaration
            var x = "bla bla";
            
             var x = "bla bla";
            
Object initialisation
            var x = new 
            {
                Name = "John",
                LastName = "Doe"
            };
            
             var x = {
                name: “John”,
                lastName: “Doe”
             };
            
Class declaration
            class MyClass : MyBaseClass
            {
                ...
            }
            
            class MyClass extends MyBaseClass {
                ...
            }
            
And many others I cannot list here.

No property declaration needed

> var myObject = new Object();
> myObject.NewProperty = "hello";
hello


Dislike

Equality operators

JavaScript has two kinds of equality operators:

  • strict equality using ===
  • loose equality using ==

Both equality compares two values for equality. The == operator is called loose because two values may be considered the same even if they are of different type. The == equality converts both values to a common type (type coercion) before comparing the values. This can potentially hide bugs and many JavaScript basic tutorial articles recommends to always use === operator. If this is the case, why bother having two equality operators in the first place? I'm sure there's a reason why the == equality is introduced, but I haven't made it my priority to look into it.


Usage of this keyword

In C# this keyword would refer to the current object. A function that wants to access the value of this should be defined inside the class. All variables in C# is lexically scoped. In JavaScript this is not always the case. The value of this is determined by how a function is called i.e. the same function will have different this value depending on how it's called.

> var myObject = new Object();
> myObject.x = 0;
>
> var myFunction = function() { 
>  this.x = 55; // what is "this" here?
> };  
>
> myObject.CallFunction = myFunction;  // Now 'myObject' has a method called 'CallFunction', which is 'myFunction'.
> myObject.CallFunction();  // This executes 'myFunction', "this" now belongs to 'myObject'
> myObject.x; 
55

Weak typing

We can add number and string together for example.

> var x = "5" + 1 + 2
512
> var y = 1 + 2 + "5" + 6
356

Here we see that if we put a number in quotes, the rest of the numbers will be treated as strings and concatenated, while the previous numbers are treated as numbers. This is actually an interesting behaviour. JavaScript is smart enough to figure out what type of data we have and then make the necessary adjustments so that we don't have to redefine it. However, coming from a strongly typed language like C#, this behaviour irks me. In a strong typed language environment, the compiler will tell me right away that I'm trying to operate on two different data types. With JavaScript, I won't get any compilation error, but I could get a different result than what I expected. I know this is a rather weak argument. Developers should know the behaviour of the language they're using, but this kind of problem seems appropriate for this blog.



Conclusion

The syntax similarity to C# in my case is unfortunately misleading because it distorts my expectations of how easy it is to switch between the two languages. This is in some way reminds me of my experience learning English and Dutch as they're quite similar to me when I first started. It's quite easy to mix both English and Dutch words in a single sentence without even realising I was doing this or thinking in English while trying to speak or construct sentences in Dutch (or the other way around). Like any language, it is up to us to learn how to use the language as it's meant to be used. We should avoid falling into the trap of thinking in one language we feel more comfortable with, while we should be thinking in the other. It is a painful process, but I believe that when we invest time to become familiar with it and stop lamenting over what it is not or what it should've been, we'll start seeing the power of the language and with experience we can easily decide which language is better suited for a specific problem we're trying to solve.

Saturday, September 19, 2015

Entity Framework cannot get Column Information for Complex Types result

This link saves me:

http://stackoverflow.com/questions/7128747/ef4-the-selected-stored-procedure-returns-no-columns

I have a stored procedure that joins multiple tables, performs queries and returns a Complex Types result. When setting this up in EF designer, EF cannot detect the columns returned from the stored procedure. As Ladislav pointed out, EF needs to execute the stored procedure to actually get the column information. I choose the second solution. I hack my stored procedure to include SET FTMONLY OFF. The first solution works too until someone tries to update the model from database or if the stored procedure is adjusted to return different columns.

The problem with this, well actually the problem with my stored procedure is that there's no default parameter value specified. EF executes my stored procedure with NULL parameter resulting in error as the stored procedure expects non NULL parameter value to be passed in, so I still ended up with no column information. I finally hack my stored procedure as follows:

IF @param1 IS NULL AND @param2 IS NULL
BEGIN
   SET FTMONLY OFF
   SET @param1 = 0
   SET @param2 = '1900-01-01'
END

@param1 is an identifier and @param2 is a date. Setting identifier to 0 and date to something way in the past guarantee that there's no result returned but that's okay since we're only interested in the column information.

Saturday, August 15, 2015

Embed SSRS Report Into WPF Application Without ReportViewer

When you google how to embed SSRS report into WPF application, chances are you'd come across this page at least once:

Walkthrough: Using ReportViewer in a WPF Application

But what if you don't want to use any of Windows Form controls? ReportViewer is Windows Form control.

There is WPF WebBrowser control we potentially could use, but it doesn't have a ready built-in attribute that we could simply set an URL to, so there's an extra step needed before we can use it. An attached property will do this for us.

    public static class WebBrowserHelper
    {
        public static readonly DependencyProperty WebAddressProperty = DependencyProperty.RegisterAttached(
            "WebAddress", 
            typeof (string), 
            typeof (WebBrowserHelper), 
            new PropertyMetadata(OnWebAddressChanged));

        public static string GetWebAddress(DependencyObject dependencyObject)
        {
            return (string) dependencyObject.GetValue(WebAddressProperty);
        }

        public static void SetWebAddress(DependencyObject dependencyObject, string value)
        {
            dependencyObject.SetValue(WebAddressProperty, value);
        }

        private static void OnWebAddressChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            WebBrowser browser = d as WebBrowser;
            if (browser != null && e.NewValue != null)
            {
                string url = e.NewValue.ToString();
                browser.Navigate(url);
            }
        }
    }

To use it, simply call the attached property from WebBrowser and bind it to the SSRS URL in the ViewModel.


<UserControl x:Class="Views.WebBrowserView"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
             xmlns:local="clr-namespace:Views.Helpers"
             mc:Ignorable="d" 
             d:DesignHeight="300" d:DesignWidth="300">   
    <Grid>
        <WebBrowser local:WebBrowserHelper.WebAddress="{Binding Path=SsrsUrl}" />
    </Grid>
</UserControl>

public class ViewModel
{
    public string SsrsUrl { get; set; }
}

And we're pretty much done.

Sunday, July 12, 2015

SQL Custom ORDER BY Multiple Columns

There're a lot of examples how to do custom ordering by one or multiple columns, but not for this particular case I'm working on. I need to do custom ordering based on a combination of static values across two tables.

   -------------------   ---------------
   | AssetClass      |   | AccountType |
   -------------------   ---------------
   | Equity          |   | FUT         |
   | Private Equity  |   | UIV         |
   | Credit          |   | NUIV        |
   | Macro           |   ---------------
   | Government Bond |
   -------------------

The query result should first be ordered by the AssetClass and then by the AccountType in the exact same order as shown above. Notice that neither is ordered alphabetically. An example of the result would look like something like this:

   ---------------------------------
   | AssetClass      | AccountType |
   ---------------------------------
   | Equity          | FUT         |        
   | Equity          | FUT         |
   | Equity          | FUT         |
   | Equity          | UIV         |
   | Equity          | UIV         |
   | Equity          | NUIV        |
   | Equity          | NUIV        |
   | Equity          | NUIV        |
   | Equity          | NUIV        |
   | Private Equity  | FUT         |
   | Private Equity  | FUT         |
   | Private Equity  | UIV         |
   | Private Equity  | UIV         |
   | Private Equity  | NUIV        |
   | Private Equity  | NUIV        |
   | ...             | ...         |
   | ...             | ...         |
   | ...             | ...         |
   | Government Bond | UIV         | 
   | Government Bond | UIV         |  
   | Government Bond | NUIV        |  
   | Government Bond | NUIV        |  
   | Government Bond | NUIV        |  
   ---------------------------------      
The ORDER BY statement is ugly but hey it does the job.
ORDER BY
    CASE [AssetClass]
        WHEN 'Equity' THEN
            CASE [AccountType]
                WHEN 'FUT' THEN 1
                WHEN 'UIV' THEN 2
                WHEN 'NUIV' THEN 3
                ELSE 4
            END
        WHEN 'Private Equity' THEN
            CASE [AccountType]
                WHEN 'FUT' THEN 5
                WHEN 'UIV' THEN 6
                WHEN 'NUIV' THEN 7
                ELSE 8
            END
        ...
        ...
        WHEN 'Government Bond' THEN
            CASE [AccountType]
                WHEN 'FUT' THEN 17
                WHEN 'UIV' THEN 18
                WHEN 'NUIV' THEN 19
                ELSE 20
            END
    END

Friday, June 12, 2015

SSRS Multi Value Parameter And Stored Procedure

I'm always struggling when it comes to dealing with SSRS multi value parameter. There're basically two ways of doing this:
1) Use SRRS Filter
2) Use good old SQL

I definitely prefer the first approach because it's so much simpler. All we need is a Dataset which returns result set for all possible values of the multi-value parameter and then we filter it based on the selected items. Let's say we have a Dataset called Companies with field Name. Steps to filter:

  1. Create a multi value parameter, let's call it @Company and suppose it contains the following items: AAA, BBB, CCC, which are the distinct names in field Name of the Dataset Companies.
  2. Click on Dataset Properties and select Filters in the left-hand pane
  3. In the Expression drop-down list, select Name as the field to filter
  4. In the Operator drop-down list box, select the In operator
  5. In the Value box, type in [@Company]
And you're done!

Now the second approach will involve creating a Stored Procedure that takes in one string argument, which contains the values of the selected multi-value parameter all joined together with a delimiter that we will need to parse in our Stored Procedure to be able to use it in the SQL IN clause. Using the same example as before, this string argument is something like: "AAA,BBB,CCC". In the SSRS report, on the Parameters of the query definition, we will need to set the parameter value to something like:

    =Join(Parameters!Company.Value,",")
This method is definitely more complicated. We will need to write a SQL string split function in our Stored Procedure taking into account the possibility that the delimiter is not always going to be a comma and whether we need to handle entry with white space in between.

Saturday, May 30, 2015

Windows Form MVVM databinding

I inherited a complex Windows Form UserControl that contains a DataGridView showing a matrix data structure like this.

The code behind takes care of the generation of the headers and cell values. It also takes care of the cell color scheme based on the value of the cell.

Because of the complexity and the fact that there's no out-of-the-box DataGridView implementation for WPF application, I decided to reuse this Windows Form UserControl in a WPF application I was working on. The problem is that Windows Form does not support data binding in WPF MVVM context. We will need to do the plumbing ourselves to be able to use the data binding functionality. One way to do this is as follows:

  1. Wrap the Windows Form UserControl in a WPF UserControl by using WindowsFormsHost
  2. Create DependencyProperty in this WPF UserControl, which we can use to bind to a viewmodel
  3. Use PropertyChangedCallback to modify property or state in the DataGridView control
  4. Use events in the DataGridView control to return property or state back to the DependencyProperty in the WPF UserControl

I'm posting a simple version of the original source code to illustrate this. Let's start with the business objects.

The Business Objects

    public class TenorStrikeRate
    {
        public TenorStrikeRate(string tenor, double strike, double rate)
        {
            Tenor = tenor;
            Strike = strike;
            Rate = rate;
        }

        public string Tenor { get; set; }

        public double Strike { get; set; }

        public double Rate { get; set; }
    }
    public class TenorStrikeRates : IEnumerable<TenorStrikeRate>
    {
        private readonly SortedList<Tuple<string, double>, TenorStrikeRate> internalData;

        public TenorStrikeRates()
        {
            internalData = new SortedList<Tuple<string, double>, TenorStrikeRate>();
        }

        public List<string> UniqueSortedTenors
        {
            get { return internalData.Values.Select(x => x.Tenor).Distinct().ToList(); }
        }

        public List<double> UniqueSortedStrikes
        {
            get { return internalData.Values.Select(x => x.Strike).Distinct().ToList(); }
        }

        public void Add(TenorStrikeRate toBeAddedItem)
        {
            Tuple<string, double> key = new Tuple<string, double>(toBeAddedItem.Tenor, toBeAddedItem.Strike);
            if (internalData.ContainsKey(key))
            {
                internalData.Remove(key);
            }

            internalData.Add(key, toBeAddedItem);
        }

        public TenorStrikeRate Find(string tenor, double strike)
        {
            return internalData.Values.FirstOrDefault(x => IsTenorEqual(x.Tenor, tenor) && IsDoubleEqual(x.Strike, strike));
        }

        public IEnumerator<TenorStrikeRate> GetEnumerator()
        {
            IEnumerator<TenorStrikeRate> iterator = internalData.Values.GetEnumerator();
            while (iterator.MoveNext())
            {
                yield return iterator.Current;
            }
        }

        IEnumerator IEnumerable.GetEnumerator()
        {
            return GetEnumerator();
        }

        public override bool Equals(object obj)
        {
            if (ReferenceEquals(null, obj))
                return false;
            if (ReferenceEquals(this, obj))
                return true;
            if (obj.GetType() != typeof(TenorStrikeRates))
                return false;

            TenorStrikeRates other = (TenorStrikeRates)obj;
            return IsAllItemInListEqual(internalData.Values, other.internalData.Values);
        }
        
        private bool IsAllItemInListEqual(IList<TenorStrikeRate> thisList, IList<TenorStrikeRate> otherList)
        {
            bool isEqual = thisList.Count.Equals(otherList.Count);
            for (int index = 0; index < otherList.Count && isEqual; index++)
            {
                TenorStrikeRate thisItem = thisList[index];
                TenorStrikeRate otherItem = otherList[index];
                isEqual = IsTenorEqual(thisItem.Tenor, otherItem.Tenor) 
                    && IsDoubleEqual(thisItem.Strike, otherItem.Strike)
                    && IsDoubleEqual(thisItem.Rate, otherItem.Rate);
            }

            return isEqual;
        }

        private bool IsDoubleEqual(double one, double two)
        {
            if (double.IsNaN(one) && double.IsNaN(two))
            {
                return true;
            }

            return Math.Abs(one - two) < 1e-15;
        }

        private bool IsTenorEqual(string one, string two)
        {
            return one.Equals(two, StringComparison.InvariantCultureIgnoreCase);
        }
    }

The Windows Form UserControl

The Windows Form UserControl contains a DataGridView named "dgv". Set(TenorStrikeRates source) populates the DataGridView. GetDataGridSource() returns the current source state of the DataGridView. In this class we also need to declare a public event to return the current data grid source to the subscribers. A situation where we want to return the current data grid source is when users perform a data grid cell editing operation. We can then hook up this public event to the DataGridView CellEndEdit event which occurs when edit mode stops for the currently selected cell.

    public partial class DataGridViewUc : System.Windows.Forms.UserControl
    {
        public DataGridViewUc()
        {
            InitializeComponent();
        }

        public delegate void ReturnGridSourceEventHandler(TenorStrikeRates currentGridSource);

        public event ReturnGridSourceEventHandler ReturnDataGridSource;

        public void Set(TenorStrikeRates source)
        {
            dgv.Columns.Clear();
            dgv.Rows.Clear();
            if (source != null)
            {
                PopulateColumnHeader(source.UniqueSortedStrikes);
                PopulateRowHeader(source.UniqueSortedTenors);
                PopulateCells(source);
            }
        }

        public TenorStrikeRates GetDataGridSource()
        {
            TenorStrikeRates quotes = new TenorStrikeRates();
            foreach (DataGridViewColumn column in dgv.Columns)
            {
                string columnHeaderValue = column.HeaderText;
                if (columnHeaderValue != null)
                {
                    double strike = ReadStrike(columnHeaderValue);
                    foreach (DataGridViewRow row in dgv.Rows)
                    {
                        object rowHeaderValue = row.HeaderCell.Value;
                        if (rowHeaderValue != null)
                        {
                            string tenor = rowHeaderValue.ToString();
                            object rateValue = row.Cells[column.Name].Value;
                            double rate = rateValue == null ? double.NaN : Convert.ToDouble(rateValue);
                            quotes.Add(new TenorStrikeRate(tenor, strike, rate));
                        }
                    }
                }
            }

            return quotes;
        }

        private void PopulateColumnHeader(List<double> strikes)
        {
            foreach (double strike in strikes)
            {
                string headerText = strike.ToString("P2");
                dgv.Columns.Add(headerText, headerText);
            }
        }

        private void PopulateRowHeader(List<string> tenors)
        {
            int numberOfRows = tenors.Count;
            dgv.Rows.Add(numberOfRows);
            for (int i = 0; i < numberOfRows; i++)
            {
                dgv.Rows[i].HeaderCell.Value = tenors[i];
            }
        }

        private void PopulateCells(TenorStrikeRates quotes)
        {
            List<string> tenors = quotes.UniqueSortedTenors;
            List<double> strikes = quotes.UniqueSortedStrikes;
            for (int rowIdx = 0; rowIdx < tenors.Count; rowIdx++)
            {
                string optionTenor = tenors[rowIdx];
                for (int colIdx = 0; colIdx < strikes.Count; colIdx++)
                {
                    double strike = strikes[colIdx];
                    TenorStrikeRate quote = quotes.Find(optionTenor, strike);
                    if (quote != null)
                    {
                        double rate = quote.Rate;
                        DataGridViewCell cell = dgv.Rows[rowIdx].Cells[colIdx];
                        cell.Value = rate;
                    }
                }
            }
        }
        
        private double ReadStrike(string input)
        {
            string percentSymbol = Thread.CurrentThread.CurrentCulture.NumberFormat.PercentSymbol;
            input = input.Replace(percentSymbol, string.Empty);
            return double.Parse(input, Thread.CurrentThread.CurrentCulture.NumberFormat) / 100;
        }

        private void dgv_CellEndEdit(object sender, DataGridViewCellEventArgs e)
        {
            if (ReturnDataGridSource != null)
            {
                TenorStrikeRates newSource = GetDataGridSource(); 
                ReturnDataGridSource(newSource);
            }
        }
    }

The WPF Host

To be able to bind the DataGridView UserControl to a viewmodel, we will need to wrap it in a WPF UserControl by using WindowsFormsHost. Then we define DependencyProperty GridSource that we can bind to a property in the viewmodel. Next we define PropertyChangedCallback OnGridSourcePropertyChanged, which is called whenever GridSource property changed. In the callback, we pass in the new source to the DataGridView so it can update its display. The WPF UserControl needs to subscribe to the DataGridView's ReturnDataGridSource event to make sure any changes in the DataGridView is propagated to the WPF host layer.

XAML:


<UserControl x:Class="DataGridViewHostedInWpfControl.WpfHost.DataGridViewUcHost"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
             xmlns:local="clr-namespace:DataGridViewHostedInWpfControl.WinFormLayer"
             mc:Ignorable="d" 
             d:DesignHeight="300" d:DesignWidth="300">
    
    <WindowsFormsHost x:Name="MyWinFormsHost">
        <local:DataGridViewUc />
    </WindowsFormsHost>

</UserControl>

Code behind:

    public partial class DataGridViewUcHost : System.Windows.Controls.UserControl
    {
        public static readonly DependencyProperty GridSourceProperty = DependencyProperty.Register(
             "GridSource",
             typeof(TenorStrikeRates),
             typeof(DataGridViewUcHost),
             new FrameworkPropertyMetadata(null, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault, OnGridSourcePropertyChanged));

        public TenorStrikeRates GridSource
        {
            get { return (TenorStrikeRates)GetValue(GridSourceProperty); }
            set { SetValue(GridSourceProperty, value); }
        }

        private static DataGridViewUc dgvUc;

        public DataGridViewUcHost()
        {
            InitializeComponent();

            dgvUc = (DataGridViewUc)MyWinFormsHost.Child;
            dgvUc.ReturnDataGridSource += OnReturnDataGridSource;
        }

        private static void OnGridSourcePropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            TenorStrikeRates newGridSource = (TenorStrikeRates)e.NewValue;
            TenorStrikeRates currentGridSource = dgvUc.GetDataGridSource();
            if (!currentGridSource.Equals(newGridSource))
            {
                dgvUc.Set(newGridSource);
            }
        }

        private void OnReturnDataGridSource(TenorStrikeRates currentGridSource)
        {
            GridSource = currentGridSource;
        }
    }

The MainWindow

Now let's build a small demo. Create a StackPanel containing the WPF control that hosts our DataGridView UserControl. Add a TextBox to the StackPanel. Everytime we change a value in the data grid, the change will be shown in the TextBox.

XAML:


<Window x:Class="DataGridViewHostedInWpfControl.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:local="clr-namespace:DataGridViewHostedInWpfControl.WpfHost"
        Title="MainWindow" Height="350" Width="525">
    
    <StackPanel>
        <Label Content="Input" />
        <local:DataGridViewUcHost GridSource="{Binding Input}" MinWidth="300" MinHeight="100" Margin="10" />
        <Label Content="Output" />
        <TextBox Text="{Binding Output}" />
    </StackPanel>

</Window>

Code behind:

    public partial class MainWindow : System.Windows.Window
    {
        public MainWindow()
        {
            MainViewModel vm = new MainViewModel();
            DataContext = vm;

            InitializeComponent();

            vm.Input = BuildInitialInput();
        }

        private TenorStrikeRates BuildInitialInput()
        {
            var quotes = new TenorStrikeRates();
            quotes.Add(new TenorStrikeRate("1y", -0.01, 0.01));
            quotes.Add(new TenorStrikeRate("1y", 0, 0.02));
            quotes.Add(new TenorStrikeRate("1y", 0.01, 0.03));
            quotes.Add(new TenorStrikeRate("2y", -0.01, 0.04));
            quotes.Add(new TenorStrikeRate("2y", 0, 0.05));
            quotes.Add(new TenorStrikeRate("2y", 0.01, 0.06));
            return quotes;
        }
    }

ViewModel:

    public class MainViewModel : INotifyPropertyChanged
    {
        private TenorStrikeRates input;
        public TenorStrikeRates Input
        {
            get { return input; }
            set
            {
                input = value;
                OnPropertyChanged("Input");
                Output = BuildOutputValue(value);
            }
        }

        private string output;
        public string Output
        {
            get { return output; }
            set
            {
                output = value;
                OnPropertyChanged("Output");
            }
        }

        public event PropertyChangedEventHandler PropertyChanged;

        protected virtual void OnPropertyChanged(string propertyName)
        {
            PropertyChangedEventHandler handler = PropertyChanged;
            if (handler != null)
            {
                handler(this, new PropertyChangedEventArgs(propertyName));
            }
        }

        private string BuildOutputValue(TenorStrikeRates tenorStrikeRates)
        {
            string text = string.Empty;
            foreach (TenorStrikeRate item in tenorStrikeRates)
            {
                text = text + string.Format("Tenor: {0}, Strike: {1}, Rate: {2}\n", item.Tenor, item.Strike.ToString("P2"), item.Rate);
            }

            return text;
        }
    }

Source Code

https://github.com/velianarie/DataGridViewHostedInWpfControl

Saturday, April 18, 2015

OpenLink Findur OpenComponents External Plugin

At work we've been trying for the last 2 years to replace our current system with OpenLink Findur. I was involved in integrating our internal pricing library by using their OpenComponents .NET API. It was tough but we successfully delivered that part of the project, so I was out of the project and had been doing other things since then. Around a month ago, I was suddenly asked to take over yet another part of this massive project from a contractor hired to do this as the management didn't want to extend her contract. So that's how I got sucked into this project again T_T.

The part I was asked to take over is a .NET C# Console Application built using OpenLink OpenComponents .NET API. This plugin is attached to Findur End of Day (EOD) workflow and runs at the very end of the workflow. This plugin extracts some of the EOD results and produces xml's needed by a lot of our legacy tools that we don't have time yet to adjust to communicate with Findur. At least not at this phase of the project.

If you're like me, I find debugging extremely important especially when you're pulled out of whatever you're doing, thrown into a massive project and expected to be up and running after 2 hours handover. Well the bad news was that she said it's impossible to debug the plugin. I asked her how she managed to build something quite big (there're more than 5000 lines of codes) without debugging it. She said she put logging lines almost everywhere, pressed F5 and waited until it either crashed (and then checked the log) or finished executing (and then checked the resulting xml's). My first reaction was: Are you kidding me?!? She also said that this plugin could only work when there's only one Findur session running in the application server as users access Findur via Citrix. At this point I had to pinch myself to check whether I was dreaming. Surely something is wrong when you need an extra application server wholly dedicated to test this plugin. It just sounds absurd.

Okay now that I'm done rambling, let's formulate the problems she mentioned and what I did to somehow make it works.

Problem #1: "You cannot debug this external Findur plugin..."


SEHException - External component has thrown an exception

So first thing I did after having all my development environment set up was putting a random breakpoint in the solution and pressing F5. It ran for a couple of seconds and then it stopped at that breakpoint. Hmmm, I thought she said we couldn't debug it? So I pressed F5 again to resume and bang! it threw a 'SEHException - External component has thrown an exception' in:

   Application app = Application.GetInstance();
Ah so this was what she meant by "You cannot debug this external Findur plugin..."

Next thing I did was going to the guy who's responsible for deploying this plugin to a test environment and asked him whether it had ever worked before and how he'd run it. He said it had worked before. At the end of Findur EOD workflow, there's a process that would trigger this plugin. He even demonstrated that it worked by manually double clicking the .exe file. I went back to my desk and double clicked the .exe file in Visual Studio bin\Debug folder. It ran successfully and produced the expected xml's. So it did seem that the contractor was right, we can't debug this plugin.

Being the person that I am, I didn't give up. To be honest I didn't want to take on a project where I'd need to code blindly and invest my time in writing log entries. Besides I wasn't even sure that log entries would help me with coding the logic of extracting data from Findur simulation results. We're talking about lots and lots of Findur Table class inquiries here. I couldn't find anything about this exception on google, nothing related to Findur, so I tried the next best thing: cleaning the solution. It had done the trick in multiple occasions in the past and I was desperate, so why not? And by "cleaning" I meant manually deleting the output files, not using Visual Studio Clean Solution. It refused to delete vshost.exe file as it's in use, obviously because I had the solution still open. Then I thought, "Wait a minute, vshost.exe is a feature in Visual Studio related to debugging and my current problem is related to debugging. Running it by double clicking the .exe works fine." At this point, I just followed my gut instinct, I closed the solution, deleted the vshost.exe, reopened the solution, recompiled and pressed F5. And then the magic happened, it didn't crash on Application.GetInstance(). Hooray!!


Debug stop working after a couple of step-into's

Apparently it's not over yet. Debugging only lasted for approximately 3 step-into's and then it simply refused to stop on any more breakpoint. The yellow line highlighting the current line went away, but it never reached the next line. The solution window title showed "(Running)" and I couldn't do anything but stopped the debugging session and restarted the whole thing. I ended up having to choose my breakpoint very wisely since I only got a limited number of step-into's. Was this what the contractor meant by "You cannot debug this external Findur plugin..."?

Again being the person that I am, I refused to give up. "Choose your breakpoints wisely" lol it even sounds silly. So going back to google search, some people recommended switching off property evaluation and other implicit function calls. This setting can be found in Tools > Options and then select Debuging > General. Find check box Enable property evaluation and other implicit function calls and uncheck it. This, my friends... didn't work.

This worked for me though:
Tools > Import and Export Settings > Reset all settings
If you're reading this because you have the same problem and about to try it out, please please note that you're going to LOSE your Resharper settings (if you're using it) and personalized keyboard shortcuts! So make sure you back up your settings when it asks you to.



Problem #2: "External plugin can only work if there's only one Findur session running..."

Original code:

   Application app = Application.GetInstance();
   Session session = app.Attach();
Application and Session class can be found in Olf.Openrisk.Application namespace. Here we see that we don't tell it to attach to a specific session. No wonder it only worked if there's only one session running. It'll just attach itself to a random session.

Now I want the plugin to attach to a specific Findur database session. If that session is not running, I want the plugin to start it and then attach itself to it. For this purpose, I introduce 3 configuration keys in App.config:

    <add key="DatabaseName" value="Findur_Test" />
    <add key="OlfBinFolder" value="C:\OpenLink\Findur_V14_1_01172015MR_02182015_1037\bin" />
    <add key="OlfParams" value="-olfcfg \\servername\ConfigFiles\TestConfig.olf -u svcAccUser -p 1234567" />
DatabaseName is the name of Findur database session we want to attach our plugin to. OlfBinFolder specifies the location where Findur is installed. There should be an executable master.exe in this bin folder which is the one we use to start Findur. OlfParams specifies the command parameters of master.exe. When you run Findur from the command prompt it reads as follows:
  C:\OpenLink\Findur_V14_1_01172015MR_02182015_1037\bin\master.exe -olfcfg \\servername\ConfigFiles\TestConfig.olf -u svcAccUser -p 1234567

Findur Application class has another Attach method where we can specify a session ID we want to attach to. There can only be one Application instance (because it's a singleton), while we can have multiple Findur sessions running on it.

If we have a Findur session running, we can see the session ID by clicking on the i (information) icon. Unfortunately this session ID changes every time we start a session and there's no way the plugin can access this information without first attaching itself to a running session, so we cannot know beforehand what the session ID is (classic chicken and egg problem isn't it? ;)). Fortunately there's SessionDescriptions property on the Application class which returns all the session ID's running on it. We can iterate through each of these session ID's, attach our plugin to it so that we can ask for the session's database name. When the attached database name is not equal to the database name we want ie. the one we specified in App.config, we detach it and continue with the next session ID and repeat the process until we find the database we want. If we don't find it, return null. Null means that the session we want to attach to is not running and therefore we will start it by running whatever specified in OlfBinFolder and OlfParams.

The adjusted code:

        Application app = Application.GetInstance();
        Session session = GetSession(app);
where:
        private Session GetSession(Application app)
        {     
            string databaseName = ConfigurationManager.AppSettings["DatabaseName"];
            Session session = GetRunningSession(app, databaseName);
            if (session == null)
            {
                string binDir = ConfigurationManager.AppSettings["OlfBinFolder"];
                string parameters = ConfigurationManager.AppSettings["OlfParams"];
                session = app.StartSession(binDir, parameters);
            }

            return session;
        }

        private Session GetRunningSession(Application app, string databaseName)
        {
            IEnumerable<int> sessionIds = app.SessionDescriptions.Select(x => x.Id);
            foreach (int sessionId in sessionIds)
            {
                Session attachedSession = app.Attach(sessionId);
                string attachedDatabaseName = attachedSession.DatabaseName;
                if (attachedDatabaseName != databaseName)
                {
                    app.Detach();
                }
                else
                {
                    return attachedSession;
                }
            }

            return null;
        }

So what is the moral of the story?
* Don't blindly trust what you're told, what does the code say?
* Don't give up and google is your friend ;)