False alarm from bots

I’m on call this week, and whenever there’s alert, it comes to my mobile. I’ve had Error rate > 5% alert from PagerDuty the other day. I’ve acknowledged the alert, (otherwise it keeps sending me the alerts) and investigated it.

A public action method ‘Login’ was not found on controller ‘xxxxWeb.Controllers.HomeController’

It was interesting. The action method definitely exists there, but we had 5 errors instantly. After investigation and some googling, it turned out that excel spreadsheet and some crawling bot were hitting the endpoint with HEAD and OPTIONS verb.

Why? I don’t know, but the fix was simple. Currently, Login was constrained to GET. Simply take off the constraint and the page will serve HEAD and OPTIONS very well.

It’s fun to be a on-call engineer. You can see loads of things you haven’t expected to see, in real world, on production server.

False alarm from bots

Dictionary.Insert Null reference error

최근에 OAuth를 이용한 로그인 시스템을 구현하면서 login client를 NuGet Package로 만들었었다. 다른 개발자들이 쉽게 로그인 기능을 이용할 수 있도록. 그런데 자꾸만 acquired token을 저장하는 static Dicionary에서 Null Reference error가 나는게 아닌가. Dictionary.Insert에서…
Dictionary.Get에서 나면 몰라도 Insert에서 Null error가 나서 쫌 구글해보니, StackOverflow의 어느 고수께서 이미 답변. Threading 이슈라고.
http://stackoverflow.com/…/how-did-i-get-this-nullreference…

그 아래 다른분은 ConcurrentDictionary를 쓰라고 친절하고 부연 설명까지.

Dictionary.Insert Null reference error

C#’s async, await, and .Result

In Market Invoice, there are many places where async and await are used. Recently, I introduced a bug that an operation gets deadlocked by replacing await with .Result. I was bitten hard:-)

Non-blocking execution

When using async and await, C# run time generates a state machine in the background

public async Task CallingMethodAsync()
{
    Task<int> longRunningTask = LongRunningOperationAsync(); // 1)
    // independent work which doesn't need the result of LongRunningOperationAsync can be done here

    //and now we call await on the task
    int result = await longRunningTask; // 2)

    //use the result
    Console.WriteLine(result);
}

public async Task<int> LongRunningOperationAsync()
{
    await Task.Delay(1000); //1 seconds delay
    return 1;
}

1) LongRunningOperationAsync is running. But it doesn’t block the execution of CallingMethodAsync, until the execution point reaches 2)

Now the execution point reached 2). If LongRunningOperationAsync() is fully done, the result will be ready, and it will be assigned to result straight away. However, if LongRunningOperationAsync() is still running, the execution of CallingMethodAsync will stop there, waiting until LongRunningOperationAsync() finishes. Once it finishes, CallingMethodAsync will resume the execution.

Call-back without its hell

Let’s look at Eric’s Serve Breakfast example.

void ServeBreakfast(Customer diner)
{
    var order = ObtainOrder(diner);
    var ingredients = ObtainIngredients(order);
    var recipe = ObtainRecipe(order);
    var meal = recipe.Prepare(ingredients);
    diner.Give(meal);
}

In this example, every customer must wait until the previous customer’s breakfast is fully prepared and served. You can see people would get angry very soon.

In order to receive orders while preparing for breakfast, you have to take orders in an asynchronous manner. It will bring it javascript’ call-back hell.

void ServeBreakfast(Diner diner)
{
  ObtainOrderAsync(diner, order =>
  {
    ObtainIngredientsAsync(order, ingredients =>
    {
      ObtainRecipeAsync(order, recipe =>
      {
        recipe.PrepareAsync(ingredients, meal =>
        {
          diner.Give(meal);
        })})})});
}

The code is not very readable. Computers may like it, but humans are not good at following up the callbacks.

This can be rewritten in the new style, reads much more nicely.

async void ServeBreakfast(Diner diner)
{
  var order = await ObtainOrderAsync(diner);
  var ingredients = await ObtainIngredientsAsync(order);
  var recipe = await ObtainRecipeAsync(order);
  var meal = await recipe.PrepareAsync(ingredients);
  diner.Give(meal);
}

Now, the methods, ObtainOrderAsync() doesn’t return order. It returns Task<Order>. It’s a callback pointer. When the execution finishes, it return the result, and order is passed into ObtainIngredientsAsync()

await or .Result

Stephen Cleary recommends using await over Result.

First, await doesn’t wrap the exception in an AggregateException, which represents one or more errors that occur during application execution. So, you will see the real exception, not the bland AggregateException. .Result wrap an exceptions that happens in the async method into AggregateException.

try {
    details = await _service.GetDetails(personId);
    ...
} catch (ApplicationException) { // this catch will work, as await pass the exception as it is.
    ...
}

For .Result, you have to catch AggregateException.

Second, Result / Wait can cause deadlocks. The async method will continue to run, and the task will be returned to it. When the task comes back, and if it’s not completed yet, it will hang in the current context.

public class CompanyDetailsController : ApiController
{
    public string Get()
    {
       var task = GetCompanyDetails(...);
       return task.Result.ToString(); // if task hasn't been completed, this will block the thread.
    }
}

public static async Task<CompanyDetails> GetCompanyDetails(Uri uri)
{
    using (var client = new HttpClient())
    {
        var jsonString = await client.GetStringAsync(uri);
        return CompanyDetails.Parse(jsonString);
    }
}

Preventing the deadlock

ConfigureAwait

await Task.Delay(1000).ConfigureAwait(
    continueOnCapturedContext: false);
  // Code here runs without the original context. (if the original context is UI thread, then UI thread context)

By using ConfigureAwait, you enable parallelism that the asynchrounous code can run in parallel with the thread the original context is in. As a result, you can avoid the deadlock

avoid Result / Wait

As Result causes deadlocks, don’t use it. Instead, favour await and use async on the method all they down or up.

 

Resources

 

C#’s async, await, and .Result

Test Data Builder with Fluent lambda interface

With the builder pattern, you can create an object in a more flexible and explicit way. And C# lamdba syntax makes the fluent interface more succinct

For example,


User fred = new UserTestDataBuilder()
.With(u => u.Name = "fred")
.With(u => u.Reputation = 900)
.With(u => u.ScholarBadge = true)
.With(u => u.CriticBadge = true)

You just need an Action<T> method and a class for the properties to populate.

public class UserSpec
{
    public string Name {get; set;}
    public int Reputation {get; set;}
    ...
}

public class UserTestDataBuilder()
{
    private UserSpec _userSpec = new UserSpec();
    public UserTestDataBuilder With(Action&lt;UserSpec&gt; action)
    {
        action(_userSpec);
        return this;
    }

    public User Build()
    {
        return new User(_userSpec.Name, _userSpec.Reputation, 
            _userSpec.ScholarBadge, _userSpec.CriticBadge);
    }
}
Test Data Builder with Fluent lambda interface

Inverting the direction of mouse scroll wheel in Windows

Why would you do that?

It’s because since in Mac OS X Lion, they’ve reversed the wheel scroll direction. I’ve always reversed it back in system preference. Then I’ve upgraded it to Yosemite and didn’t bother to change it. I just change myself to follow it. A few days later, the Mac mouse scroll wheel direction became natural to me.

Now I have a problem with my work PC, which is Windows Server 2012. Whenever I scroll the page with the mouse, the page goes in the opposite direction and it really annoys and confuses me.

So, now you need to reverse the mouse wheel scroll in windows. How can you do that?

You have to edit a registry, unfortunately, but there’s a powershell script that does it for you. Of course, you shouldn’t run any script you find on Internet in your admin elevated powershell shell. But I took the risk and did it. This is the script.

Get-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Enum\HID\*\*\Device` Parameters FlipFlopWheel -EA 0 | ForEach-Object { Set-ItemProperty $_.PSPath FlipFlopWheel 1 }

Good luck and enjoy the inverted direction of the mouse scroll wheel.

Inverting the direction of mouse scroll wheel in Windows

Reactive Extensions, to write async, event-based programs with observables

Hello World in Reactive way

Reactive Programming is “a programming paradigm oriented around data flows and the propagation of change” (wikipedia)

With Reactive Extensions (Rx), you can write asynchronous and event-based programs using observable sequences. Rx let you represent asynchronous data streams with Observables, (push-based notifications) and query asynchronous data streams using LINQ, Simply put “Rx = Observables + LINQ + Schedulers”.

You can install the package via nuget.

pm> Install-Package Rx-Main

Channel9 has a concise introduction video: Rx Workshop Introduction. The simplest “Hello, World” can be done in this way.

class Program
{
    static void Main(string[] args)
    {
        var streamOfChars = "Hello, World".ToObservable();
        streamOfChars.Subscribe(c => Console.WriteLine(c));
    }
}

Another simple example is to enumerate from 1 to 10 and subscribe to it.

IObservable<int> source = Observable.Range(1, 10);
IDisposable subscription = source.Subscribe(
   x => Console.WriteLine("OnNext: {0}", x),
   ex => Console.WriteLine("OnError: {0}", ex.Message),
   () => Console.WriteLine("OnCompleted"));

Console.WriteLine("Press ENTER to unsubscribe...");
Console.ReadLine();
subscription.Dispose();

Cold vs. Hot Observables

Cold observables start running on subscription, that is, it starts pushing values to the observables when Subscribe is called. This doesn’t fit in the real-world case, like stock tickers, which should be producing values even before a subscription is active. The observer that subscribes to a hot observable sequence, get the current value in the stream.

Console.WriteLine("Current Time: " + DateTime.Now);
var source = Observable.Interval(TimeSpan.FromSeconds(1));
//creates a sequence

IConnectableObservable<long> hot = Observable.Publish<long>(source);  
// convert the sequence into a hot sequence

IDisposable subscription1 = hot.Subscribe(                        
    x => Console.WriteLine("Observer 1: OnNext: {0}", x),
    ex => Console.WriteLine("Observer 1: OnError: {0}", ex.Message),
    () => Console.WriteLine("Observer 1: OnCompleted"));
Console.WriteLine("Current Time after 1st subscription: " + DateTime.Now);
Thread.Sleep(3000);  //idle for 3 seconds
hot.Connect();       
// hot is connected to source and starts pushing value to subscribers 

Console.WriteLine("Current Time after Connect: " + DateTime.Now);
Thread.Sleep(3000);  //idle for 3 seconds
Console.WriteLine("Current Time just before 2nd subscription: " + DateTime.Now);

// value will immediately be pushed to 2nd subscription
IDisposable subscription2 = hot.Subscribe(     
    x => Console.WriteLine("Observer 2: OnNext: {0}", x),
    ex => Console.WriteLine("Observer 2: OnError: {0}", ex.Message),
    () => Console.WriteLine("Observer 2: OnCompleted"));
Console.ReadKey();

An example from MSDN.

Reactive Extensions, to write async, event-based programs with observables

git cherry-pick

Git commit’s id is a hash of its contents and its history, and becomes a unique id for a specific commit. Even if it contains the same change, as the parent would be different, it’ll have a different id.

“git cherry-pick” takes a commit from somewhere else, and “play it back” where you are right now. Git will build a new commit with a different hash, as the parents are different, though the contents are the same. One thing to note is that git hash is branch agnostic. in Git, an branch is simply “a lightweight movable pointer to one of these commits” (from git branching)

The other day, I rebuilt a release branch for release (of course!). I had to fix one issue, so committed the fix to the release branch. I made a few other changes and revoked the change, as they were not really necessary. Now I wanted to cheery-pick the commit for the fix.

I did git log.

C:\Users\andrew.chaa\Documents\Projects\PopOpen [release]> git log
commit 36bfde24c821f36f84c6ec88c796ae6edac17286
Author: andrewchaa <>
Date: Wed May 13 15:45:34 2015 +0100
the version is updated to 0.8.6

commit 8c803f203a03b9bd11faca7a754e0f1f4c8ab1b3
Author: andrewchaa <>
Date: Tue May 5 11:23:34 2015 +0100
Added Logging. No more fixed time looping. Use MainWindowTitle and check if the found window has the same fil

I know the commit hash, 8c803f203a03b9bd11faca7a754e0f1f4c8ab1b3.

git checkout master
git cherry-pick 8c803f203a03b9bd11faca7a754e0f1f4c8ab1b3

Then the change is on top of the last commit on the master branch. I can push the change to the server.

git cherry-pick