Delivery PipeLine Approach (CoD)

Usually there are many projects waiting in the delivery queue, in most cases the HiPPO method is what determines their relative priority and thus the order they are implemented in. Pet projects take precedence on potentially more important changes, with no consideration to the financial impact especially given how illusive it is to measure the financial impact of unrealised projects.

cartoon-hippo-clipart-1

HiPPO

According to the Lean Enterprise, the biggest two risks to any IT projects are: Projects getting cancelled halfway through and Delivered projects to be shelved and never used. Based on my experience both are quite common, perhaps because of the aforementioned project prioritisation methods.

The Cost of Delay method (CoD) is an analytical approach that can help prioritise projects based on a measurable metric that can be understood by business. Simple, perhaps simplistic even yet it is much more scientific than the HiPPO approach.

CoD works as follows (and I’m using an example from the Lean Enterprise book), first each project is assigned a cost of delay value, for instance Project A which is a compliance project would entail a penalty of $250k per week of none compliance. Project B introduces a new sales method which is expected to bring in $100k per week. There are three options: A is implemented first, B implemented first, A&B are implemented concurrently.

 

Screen Shot 2016-08-27 at 9.34.53 PM

Scenario 1: If A is implemented first , the cost of delay incurred by delaying B is $200k.

Scenario 2: If B is implemented first, the cost of delaying A is $250k.

Scenario 3: If A&B are concurrently implemented with the resources split between them, the cost of delay is $350K

Based on this the decision is easy, implementing A first has a lower CoD hence it is the best choice to go for. An objective rational decision that doesn’t rely on personal preferences of product owners.

Real life scenarios are more complicated, as calculating the CoD for a project often includes many factors some of which are not foreseeable, Furthermore the CoD is often not constant over time and risk might increase overtime with increase in sales. Even more interesting projects are often not independent and delaying one project might result in a chain of events incurring unpredictable costs.

CoD is not perfect yet it is provides an economical, analytical way to prioritise projects within a complex pipeline.

 

 

 

Downloading Torrents Remotely

I’ve set up a home server and wanting a simple straight forward way to download torrents remotely I relied on an old hack I’ve heard about but never attempted. You can configure transmission the torrents client to pick up torrents files from a certain directory, sharing this directory on google drive means that you can drop torrent files for transmission to download for you at home.

Screen Shot 2016-03-15 at 4.21.56 PM.png

The problem is you can’t really monitor the progress of the torrents and some of these torrent files may not even start. So I decided to write a small shell script that monitors two events and updates me through push bullet, the first event being the torrent download start (creation of a *.part file) the second event is the completion of the download (new file in the downloads directory).

The script works as follows:

Screen Shot 2016-03-15 at 4.15.30 PM.png

 
#!/bin/bash
partsLocation=<DOWNLOADS LOCATION HERE>
completeLocation=<COMPLETE LOCATION HERE>
logLocation=./TorrentsMonitor.log
pushBulletAPI=<PBAPI KEY HERE>

for i in $(find $partsLocation -name "*.part" -maxdepth 1|sed -e "s/ /_/g");
do 
echo "Handling File"
echo $i;
echo "----------------------"

#apply check here if file exists
countOfParts=$(cat parts.log|grep $i|wc -l)
echo $countOfParts
if [ $countOfParts -gt 0 ]
then 
echo "already listed"
else 
echo "new file, adding to parts.log"
echo $i >> parts.log
curl -u $pushBulletAPI: https://api.pushbullet.com/v2/pushes -d type=note -d title="Tor Started" -d body="Download started for file $i"
fi
done

#-------------------------
#scan for complete files
#-----------------------------

for i in $(find $completeLocation ! -name '*.part' ! -name '*.log' ! -name "*.sh*" ! -name "." -maxdepth 1|sed -e "s/ /_/g");
do 
echo "Handling File"
echo $i;
echo "----------------------"
#apply check here if file exists
countOfComplete=$(cat complete.log|grep $i|wc -l)
echo $countOfComplete
if [ $countOfComplete -gt 0 ]
then 
echo "already listed"
else 
echo "new file, adding to complete.log"
echo $i >> complete.log

curl -u $pushBulletAPI: https://api.pushbullet.com/v2/pushes -d type=note -d title="Tor Completed" -d body="Download Completed for file $i"
fi
done

Deploying A Mobile APP in an Enterprise Environment

The use of mobile application on consumer grade devices is increasing in popularity as more and more companies are using customised apps on mobile devices for field purposes instead of using a purpose built device. Example of such an implementation can range from Biometrics Scanning, Merchandise Delivery or even Taxi Dispatching.

BlogImage_E-ioT.2

Certain risks are associated with this approach since unlike web applications each user is responsible for managing and updating his version, just like the pre web-apps days back when people used desktop applications, These risks include:

  1. Using an obsolete version of the app that is no longer compatible with the backend.
  2. Using a version of the app that include a critical security issue.
  3. Incorrect business process due to the use of an older version of the app.
  4. Using a none official version of the app using the same backend, thus bypassing any front end validations.

There are certain guidelines that can be followed to control the inherent risk, I’m going to list some of them here as a guideline

I. Upgrade Enforcement 

For critical upgrades that renders the previous versions obsolete, for instance changing the business process or introducing a critical security enhancement. The best practice is to break the backends backward compatibility.

Breaking the backend backward compatibility can be done by adding an app version check with every request, including the app version in every request is easy and has a negligible cost on both data and processing yet is very useful when needed. The server response should include an error code that would trigger a “You Must Upgrade Now” message.

II. Upgrade Notification

For less critical updates push notifications can be used to suggest an upgrade to the customer, a more aggressive approach  (for android) would be to handle the push to fire the play store on the application’s URI. The frequency of these notifications can reflect how important this update is.

III. Application Verification

To restrict against none official apps the api should include a verification token, there are many ways to implement this, one of the easiest way would be encrypting one of the fields (timestamp for instance) and sending both the encrypted as well as the none encrypted versions. The backend would then verify the version of the app by comparing the decrypted field against the plain text one, if they do not match the response should indicate that.

There are many ways to implement this, the encrypted field approach happens to be the easiest way to do it.

IV. Root Check/Emulator Check

Rooted phones can offer a malicious user the means to manipulate the backend calls, while keeping the verification field. A root check can be conducted on the device every time an activity is started. Emulators are easy to uncover as well.

V. Malicious Usage checks

Just in case all of your checks fail, backend should conduct even a rudimentary malicious behaviour check, blocking devices that exhibit none expected behaviour.

VI. Connectivity Issues

Even with the advances in cellular service coverage 3G/4G service remains spotty especially in rural areas. There are few ways to mitigate this depending on the nature of the requirements. If no online/sync operations are required you can implement a simple request cashing service, in which server side requests are cashed to be retried when connectivity is available.

VII. Usage Patterns Analytics (for android)

Creating usage heat maps is important when it comes to determining how the people are actually using the applications and whether certain features should be augmented or removed due to lack of usage. Luckily google analytics can be integrated to track usage or activity launches, it can even be used to track individual controls actions.

 

I hope this post was helpful I’m planning to write another post soon on how to conduct unit, QA and scale tests on enterprise apps.

Extracting DM Images Over Twitter API

twittercloud

Extracting images from received DMs over twitter API has proven to be rather tricky and there aren’t enough information on how to do it. In this entry I’m going to explain how to achieve that programatically, using both curl as well as twitter4j.

*Note: you have to make sure that the app you are using has “Read, Write and Access direct messages” permission, other wise you’ll get a “HTTP/1.1 401 Authorization Required” error. 

Step 1: Get the Image ton URL

Within DMs images are represented as media entities, the media entity has several links, the one you need is “media_url”. the https://ton.* link

Screen Shot 2015-10-25 at 11.30.25 AM

 Step2 : Use the link to download the image 

Using Twitter4j: 

Call twitter.getDMImageAsStream(“TON URL“), this will return an input stream of the image.

Using Curl:

Put the URL in the get command with the standard twitter OAuth headers. Screen Shot 2015-10-25 at 11.38.08 AM

Particle Photon Powered Physical Notifications Device

wpid-20151023_021501.jpg

Having just received my photon maker kit I wanted to build a simple project to show case the photon’s capabilities and so I decided to build a photon powered RGB lantern which I’d light up using IFTTT recipes.

image

The Internet Powered Lamp Lighting Up

The components :

  • Particle Photon
  • 3 x 8mm RGB LEDs (common anode)
  • 330ohm resistor
  • Prototyping board
  • Battery Pack
  • Ikea Paper Lantern

Wiring:
image

  • Connect the 3 RGB in parallel with the  the resistor connected to the common anode.
  • Red to A4
  • Green to A3
  • Red to A2
  • Common anode resistor to D0
  • Battery pack positive to photon Vin
  • Battery pack -Ve to photon GND

Code:

I didn’t want the lantern to light up all of a sudden instead I wanted it to gradually light up.

// -----------------------------------
// Controlling RGB LEDs over the Internet
// 23 October 2015
// -----------------------------------
int led_red = A5;
int led_blue = A4;
int led_green = A3;
int led_on = D0;

int led=D7;

int r_val=255;
int g_val=255;
int b_val=255;
int p_val=0;
int t_val=1;
bool is_on=false;
// Last time, we only needed to declare pins in the setup function.
// This time, we are also going to register our Spark function

void setup()
{
    Serial.begin(9600); 
  
  
   // Pin configuration, 3 PINS, plus anode
   pinMode(led_on, OUTPUT);
   pinMode(led_red, OUTPUT);
   pinMode(led_blue, OUTPUT);
   pinMode(led_green, OUTPUT);

   digitalWrite(led,HIGH);
    
    
    //we declare a photon function to use through IFTTT
   Spark.function("led",ledToggle);
  
    //turn the LEDs off 
   digitalWrite(led_on,LOW);
   analogWrite(led_green,255);
   analogWrite(led_red,255);
   analogWrite(led_blue,255);

}

void loop()
{
   // Nothing to do here
}

/*This is the function IFTTT will be talking to, it accepts a string command
*/

int ledToggle(String command) {
    /* Spark function to receive colour and duration RRRGGGBBBTTT example 000255255001 for RED turn on for 1 second, 
    values are reversed so 000 is the strongest illumination and 255 is off
    */

    is_on=true;
    /*debug bit*/
    Serial.println("c : ");
    Serial.println("command recieved "+command);
    
    /*tokenize string and extract colors*/
    String red=command.substring(0,3);
    String green=command.substring(3,6);
    String blue=command.substring(6,9);
    String backoff_t=command.substring(9,12);
    
    /*debug bit*/
    Serial.println("R"+red);
    Serial.println("G"+green);
    Serial.println("B"+blue);
    Serial.println("t"+backoff_t);

    /*cast string colors*/
    r_val=red.toInt();
    g_val=green.toInt();
    b_val=blue.toInt();
    t_val=backoff_t.toInt();

/*cycle colours on*/
    set_lights();
    
    /*keep lights on for the required duration*/
    delay(1000*t_val);
    
    //reset vars and turn off 
    is_on=false;
    r_val=255;
    g_val=255;
    b_val=255;
    t_val=1;
    
    set_lights();
}

/*
Function used to set lights
*/
int set_lights()
{
    //colour cycle while lighting up
    
    //set colour
    analogWrite(led_red,r_val);
    analogWrite(led_green,g_val);
    analogWrite(led_blue,b_val);
    
    if(is_on)
    {
        // gradual power increase to common anode
        for(int i =0; i<=255; i++)
        {
             delay(5);
             analogWrite(led_on,i);
        }
    }
    else 
    {
        analogWrite(led_on,0);
    }
   
}

IFTTT integration:

Once the code was flashed on my photon, I was able to invoke the photon function over the internet, with the desired RGB colour value and duration sent as a parameter.

IFTTT Action

IFTTT Action, to light up purple for 20 seconds

 

 
**Update: I cleaned up the implementation by using a neopixel stick which greatly reduced the wiring and increased the functionality**