Command-Line app with Node.js

JavaScript has grown massively recently with the introduction of Node.js, and it serves as a language for Front-End, Server-side api, desktop app, and even for mobile app. It’s a kind of universal language every platform understands.

So, it’s no wonder that you build command-line tools with node.js. Recently, at Just-Eat, I had to upload a custom ZenDesk app via ZenDesk api and wanted to build a script that would do the job. So my choice was node.js with command.js.

Packages you need

With command.js, you can build command-line arguments in a very clear way.

const program = require('commander');
const request = require('request');
const fs = require('fs');

console.log('ex) node zen i <subdomain> <token>\n');

  .description('ZenDial Installer');

  .command('install <subdomain> <token>')
  .description('to install ZenDial')
  .action((subdomain, token) => {
    install({subdomain, token});

One tricky bit was to add basic authentication with formData. I’ve spend 20-30 minutes to figure it out. It turned out that I had to specify the endpoint url in {} object.

const install = (params) => {

  const user = '';
  const password = params.token;
  const fileStream = fs.createReadStream('');

  console.log('uploading the file...'){
    url: 'https://' + params.subdomain + '',
    formData: { uploaded_data: fileStream },
    auth: { 'username': user, 'password': password }
  }, function(err, res, body) {
    if (err) {
      console.error('upload failed:', err);

    console.log('file uploaded')

    const uploadId = JSON.parse(body).id;
    console.log('upload id: ' + uploadId){
      url: 'https://' + params.subdomain + '',
      form: {
        name: 'Zendial v2',
        short_description: 'Zendesk-to-liveops integration app',
        upload_id: uploadId
      json: true,
      auth: { 'username': user, 'password': password }
    }, function (err, res, body) {
        if (err) console.log(err);




False alarm from bots

I’m on call this week, and whenever there’s alert, it comes to my mobile. I’ve had Error rate > 5% alert from PagerDuty the other day. I’ve acknowledged the alert, (otherwise it keeps sending me the alerts) and investigated it.

A public action method ‘Login’ was not found on controller ‘xxxxWeb.Controllers.HomeController’

It was interesting. The action method definitely exists there, but we had 5 errors instantly. After investigation and some googling, it turned out that excel spreadsheet and some crawling bot were hitting the endpoint with HEAD and OPTIONS verb.

Why? I don’t know, but the fix was simple. Currently, Login was constrained to GET. Simply take off the constraint and the page will serve HEAD and OPTIONS very well.

It’s fun to be a on-call engineer. You can see loads of things you haven’t expected to see, in real world, on production server.

Dictionary.Insert Null reference error

최근에 OAuth를 이용한 로그인 시스템을 구현하면서 login client를 NuGet Package로 만들었었다. 다른 개발자들이 쉽게 로그인 기능을 이용할 수 있도록. 그런데 자꾸만 acquired token을 저장하는 static Dicionary에서 Null Reference error가 나는게 아닌가. Dictionary.Insert에서…
Dictionary.Get에서 나면 몰라도 Insert에서 Null error가 나서 쫌 구글해보니, StackOverflow의 어느 고수께서 이미 답변. Threading 이슈라고.…/how-did-i-get-this-nullreference…

그 아래 다른분은 ConcurrentDictionary를 쓰라고 친절하고 부연 설명까지.

C#’s async, await, and .Result

In Market Invoice, there are many places where async and await are used. Recently, I introduced a bug that an operation gets deadlocked by replacing await with .Result. I was bitten hard 🙂

Non-blocking execution

When using async and await, C# run time generates a state machine in the background

public async Task CallingMethodAsync()
    Task<int> longRunningTask = LongRunningOperationAsync(); // 1)
    // independent work which doesn't need the result of LongRunningOperationAsync can be done here

    //and now we call await on the task
    int result = await longRunningTask; // 2)

    //use the result

public async Task<int> LongRunningOperationAsync()
    await Task.Delay(1000); //1 seconds delay
    return 1;

1) LongRunningOperationAsync is running. But it doesn’t block the execution of CallingMethodAsync, until the execution point reaches 2)

Now the execution point reached 2). If LongRunningOperationAsync() is fully done, the result will be ready, and it will be assigned to result straight away. However, if LongRunningOperationAsync() is still running, the execution of CallingMethodAsync will stop there, waiting until LongRunningOperationAsync() finishes. Once it finishes, CallingMethodAsync will resume the execution.

Call-back without its hell

Let’s look at Eric’s Serve Breakfast example.

void ServeBreakfast(Customer diner)
    var order = ObtainOrder(diner);
    var ingredients = ObtainIngredients(order);
    var recipe = ObtainRecipe(order);
    var meal = recipe.Prepare(ingredients);

In this example, every customer must wait until the previous customer’s breakfast is fully prepared and served. You can see people would get angry very soon.

In order to receive orders while preparing for breakfast, you have to take orders in an asynchronous manner. It will bring it javascript’ call-back hell.

void ServeBreakfast(Diner diner)
  ObtainOrderAsync(diner, order =>
    ObtainIngredientsAsync(order, ingredients =>
      ObtainRecipeAsync(order, recipe =>
        recipe.PrepareAsync(ingredients, meal =>

The code is not very readable. Computers may like it, but humans are not good at following up the callbacks.

This can be rewritten in the new style, reads much more nicely.

async void ServeBreakfast(Diner diner)
  var order = await ObtainOrderAsync(diner);
  var ingredients = await ObtainIngredientsAsync(order);
  var recipe = await ObtainRecipeAsync(order);
  var meal = await recipe.PrepareAsync(ingredients);

Now, the methods, ObtainOrderAsync() doesn’t return order. It returns Task<Order>. It’s a callback pointer. When the execution finishes, it return the result, and order is passed into ObtainIngredientsAsync()

await or .Result

Stephen Cleary recommends using await over Result.

First, await doesn’t wrap the exception in an AggregateException, which represents one or more errors that occur during application execution. So, you will see the real exception, not the bland AggregateException. .Result wrap an exceptions that happens in the async method into AggregateException.

try {
    details = await _service.GetDetails(personId);
} catch (ApplicationException) { // this catch will work, as await pass the exception as it is.

For .Result, you have to catch AggregateException.

Second, Result / Wait can cause deadlocks. The async method will continue to run, and the task will be returned to it. When the task comes back, and if it’s not completed yet, it will hang in the current context.

public class CompanyDetailsController : ApiController
    public string Get()
       var task = GetCompanyDetails(...);
       return task.Result.ToString(); // if task hasn't been completed, this will block the thread.

public static async Task<CompanyDetails> GetCompanyDetails(Uri uri)
    using (var client = new HttpClient())
        var jsonString = await client.GetStringAsync(uri);
        return CompanyDetails.Parse(jsonString);

Preventing the deadlock


await Task.Delay(1000).ConfigureAwait(
    continueOnCapturedContext: false);
  // Code here runs without the original context. (if the original context is UI thread, then UI thread context)

By using ConfigureAwait, you enable parallelism that the asynchrounous code can run in parallel with the thread the original context is in. As a result, you can avoid the deadlock

avoid Result / Wait

As Result causes deadlocks, don’t use it. Instead, favour await and use async on the method all they down or up.




Test Data Builder with Fluent lambda interface

With the builder pattern, you can create an object in a more flexible and explicit way. And C# lamdba syntax makes the fluent interface more succinct

For example,

User fred = new UserTestDataBuilder()
.With(u => u.Name = "fred")
.With(u => u.Reputation = 900)
.With(u => u.ScholarBadge = true)
.With(u => u.CriticBadge = true)

You just need an Action<T> method and a class for the properties to populate.

public class UserSpec
    public string Name {get; set;}
    public int Reputation {get; set;}

public class UserTestDataBuilder()
    private UserSpec _userSpec = new UserSpec();
    public UserTestDataBuilder With(Action&lt;UserSpec&gt; action)
        return this;

    public User Build()
        return new User(_userSpec.Name, _userSpec.Reputation, 
            _userSpec.ScholarBadge, _userSpec.CriticBadge);

Inverting the direction of mouse scroll wheel in Windows

Why would you do that?

It’s because since in Mac OS X Lion, they’ve reversed the wheel scroll direction. I’ve always reversed it back in system preference. Then I’ve upgraded it to Yosemite and didn’t bother to change it. I just change myself to follow it. A few days later, the Mac mouse scroll wheel direction became natural to me.

Now I have a problem with my work PC, which is Windows Server 2012. Whenever I scroll the page with the mouse, the page goes in the opposite direction and it really annoys and confuses me.

So, now you need to reverse the mouse wheel scroll in windows. How can you do that?

You have to edit a registry, unfortunately, but there’s a powershell script that does it for you. Of course, you shouldn’t run any script you find on Internet in your admin elevated powershell shell. But I took the risk and did it. This is the script.

Get-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Enum\HID\*\*\Device` Parameters FlipFlopWheel -EA 0 | ForEach-Object { Set-ItemProperty $_.PSPath FlipFlopWheel 1 }

Good luck and enjoy the inverted direction of the mouse scroll wheel.

Reactive Extensions, to write async, event-based programs with observables

Hello World in Reactive way

Reactive Programming is “a programming paradigm oriented around data flows and the propagation of change” (wikipedia)

With Reactive Extensions (Rx), you can write asynchronous and event-based programs using observable sequences. Rx let you represent asynchronous data streams with Observables, (push-based notifications) and query asynchronous data streams using LINQ, Simply put “Rx = Observables + LINQ + Schedulers”.

You can install the package via nuget.

pm> Install-Package Rx-Main

Channel9 has a concise introduction video: Rx Workshop Introduction. The simplest “Hello, World” can be done in this way.

class Program
    static void Main(string[] args)
        var streamOfChars = "Hello, World".ToObservable();
        streamOfChars.Subscribe(c => Console.WriteLine(c));

Another simple example is to enumerate from 1 to 10 and subscribe to it.

IObservable<int> source = Observable.Range(1, 10);
IDisposable subscription = source.Subscribe(
   x => Console.WriteLine("OnNext: {0}", x),
   ex => Console.WriteLine("OnError: {0}", ex.Message),
   () => Console.WriteLine("OnCompleted"));

Console.WriteLine("Press ENTER to unsubscribe...");

Cold vs. Hot Observables

Cold observables start running on subscription, that is, it starts pushing values to the observables when Subscribe is called. This doesn’t fit in the real-world case, like stock tickers, which should be producing values even before a subscription is active. The observer that subscribes to a hot observable sequence, get the current value in the stream.

Console.WriteLine("Current Time: " + DateTime.Now);
var source = Observable.Interval(TimeSpan.FromSeconds(1));
//creates a sequence

IConnectableObservable<long> hot = Observable.Publish<long>(source);  
// convert the sequence into a hot sequence

IDisposable subscription1 = hot.Subscribe(                        
    x => Console.WriteLine("Observer 1: OnNext: {0}", x),
    ex => Console.WriteLine("Observer 1: OnError: {0}", ex.Message),
    () => Console.WriteLine("Observer 1: OnCompleted"));
Console.WriteLine("Current Time after 1st subscription: " + DateTime.Now);
Thread.Sleep(3000);  //idle for 3 seconds
// hot is connected to source and starts pushing value to subscribers 

Console.WriteLine("Current Time after Connect: " + DateTime.Now);
Thread.Sleep(3000);  //idle for 3 seconds
Console.WriteLine("Current Time just before 2nd subscription: " + DateTime.Now);

// value will immediately be pushed to 2nd subscription
IDisposable subscription2 = hot.Subscribe(     
    x => Console.WriteLine("Observer 2: OnNext: {0}", x),
    ex => Console.WriteLine("Observer 2: OnError: {0}", ex.Message),
    () => Console.WriteLine("Observer 2: OnCompleted"));

An example from MSDN.