False alarm from bots

I'm on call this week, and whenever there's alert, it comes to my mobile. I've had Error rate > 5% alert from PagerDuty the other day. I've acknowledged the alert, (otherwise it keeps sending me the alerts) and investigated it.

A public action method 'Login' was not found on controller 'xxxxWeb.Controllers.HomeController'

It was interesting. The action method definitely exists there, but we had 5 errors instantly. After investigation and some googling, it turned out that excel spreadsheet and some crawling bot were hitting the endpoint with HEAD and OPTIONS verb.

Why? I don't know, but the fix was simple. Currently, Login was constrained to GET. Simply take off the constraint and the page will serve HEAD and OPTIONS very well.

It's fun to be a on-call engineer. You can see loads of things you haven't expected to see, in real world, on production server.

Dictionary.Insert Null reference error

최근에 OAuth를 이용한 로그인 시스템을 구현하면서 login client를 NuGet Package로 만들었었다. 다른 개발자들이 쉽게 로그인 기능을 이용할 수 있도록. 그런데 자꾸만 acquired token을 저장하는 static Dicionary에서 Null Reference error가 나는게 아닌가. Dictionary.Insert에서...
Dictionary.Get에서 나면 몰라도 Insert에서 Null error가 나서 쫌 구글해보니, StackOverflow의 어느 고수께서 이미 답변. Threading 이슈라고.
http://stackoverflow.com/…/how-did-i-get-this-nullreference…

그 아래 다른분은 ConcurrentDictionary를 쓰라고 친절하고 부연 설명까지.

C#'s async, await, and .Result

In Market Invoice, there are many places where async and await are used. Recently, I introduced a bug that an operation gets deadlocked by replacing await with .Result. I was bitten hard :-)

Non-blocking execution

When using async and await, C# run time generates a state machine in the background

[sourcecode language="csharp"]
public async Task CallingMethodAsync()
{
Task<int> longRunningTask = LongRunningOperationAsync(); // 1)
// independent work which doesn't need the result of LongRunningOperationAsync can be done here

//and now we call await on the task
int result = await longRunningTask; // 2)

//use the result
Console.WriteLine(result);
}

public async Task<int> LongRunningOperationAsync()
{
await Task.Delay(1000); //1 seconds delay
return 1;
}
[/sourcecode]

1) LongRunningOperationAsync is running. But it doesn't block the execution of CallingMethodAsync, until the execution point reaches 2)

Now the execution point reached 2). If LongRunningOperationAsync() is fully done, the result will be ready, and it will be assigned to result straight away. However, if LongRunningOperationAsync() is still running, the execution of CallingMethodAsync will stop there, waiting until LongRunningOperationAsync() finishes. Once it finishes, CallingMethodAsync will resume the execution.

Call-back without its hell

Let's look at Eric's Serve Breakfast example.

[sourcecode language="csharp"]
void ServeBreakfast(Customer diner)
{
var order = ObtainOrder(diner);
var ingredients = ObtainIngredients(order);
var recipe = ObtainRecipe(order);
var meal = recipe.Prepare(ingredients);
diner.Give(meal);
}
[/sourcecode]

In this example, every customer must wait until the previous customer's breakfast is fully prepared and served. You can see people would get angry very soon.

In order to receive orders while preparing for breakfast, you have to take orders in an asynchronous manner. It will bring it javascript' call-back hell.

[sourcecode language="csharp"]
void ServeBreakfast(Diner diner)
{
ObtainOrderAsync(diner, order =>
{
ObtainIngredientsAsync(order, ingredients =>
{
ObtainRecipeAsync(order, recipe =>
{
recipe.PrepareAsync(ingredients, meal =>
{
diner.Give(meal);
})})})});
}
[/sourcecode]

The code is not very readable. Computers may like it, but humans are not good at following up the callbacks.

This can be rewritten in the new style, reads much more nicely.

[sourcecode language="csharp"]
async void ServeBreakfast(Diner diner)
{
var order = await ObtainOrderAsync(diner);
var ingredients = await ObtainIngredientsAsync(order);
var recipe = await ObtainRecipeAsync(order);
var meal = await recipe.PrepareAsync(ingredients);
diner.Give(meal);
}
[/sourcecode]

Now, the methods, ObtainOrderAsync() doesn't return order. It returns Task<Order>. It's a callback pointer. When the execution finishes, it return the result, and order is passed into ObtainIngredientsAsync()

await or .Result

Stephen Cleary recommends using await over Result.

First, await doesn't wrap the exception in an AggregateException, which represents one or more errors that occur during application execution. So, you will see the real exception, not the bland AggregateException. .Result wrap an exceptions that happens in the async method into AggregateException.

[sourcecode language="csharp"]
try {
details = await _service.GetDetails(personId);
...
} catch (ApplicationException) { // this catch will work, as await pass the exception as it is.
...
}
[/sourcecode]

For .Result, you have to catch AggregateException.

Second, Result / Wait can cause deadlocks. The async method will continue to run, and the task will be returned to it. When the task comes back, and if it's not completed yet, it will hang in the current context.

[sourcecode language="csharp"]
public class CompanyDetailsController : ApiController
{
public string Get()
{
var task = GetCompanyDetails(...);
return task.Result.ToString(); // if task hasn't been completed, this will block the thread.
}
}

public static async Task<CompanyDetails> GetCompanyDetails(Uri uri)
{
using (var client = new HttpClient())
{
var jsonString = await client.GetStringAsync(uri);
return CompanyDetails.Parse(jsonString);
}
}
[/sourcecode]

Preventing the deadlock

ConfigureAwait

[sourcecode language="csharp"]
await Task.Delay(1000).ConfigureAwait(
continueOnCapturedContext: false);
// Code here runs without the original context. (if the original context is UI thread, then UI thread context)
[/sourcecode]

By using ConfigureAwait, you enable parallelism that the asynchrounous code can run in parallel with the thread the original context is in. As a result, you can avoid the deadlock

avoid Result / Wait

As Result causes deadlocks, don't use it. Instead, favour await and use async on the method all they down or up.

 

Resources

 

Test Data Builder with Fluent lambda interface

With the builder pattern, you can create an object in a more flexible and explicit way. And C# lamdba syntax makes the fluent interface more succinct

For example,

[sourcecode language="csharp"]

User fred = new UserTestDataBuilder()
.With(u => u.Name = "fred")
.With(u => u.Reputation = 900)
.With(u => u.ScholarBadge = true)
.With(u => u.CriticBadge = true)

[/sourcecode]

You just need an Action<T> method and a class for the properties to populate.

[sourcecode language="csharp"]
public class UserSpec
{
public string Name {get; set;}
public int Reputation {get; set;}
...
}

public class UserTestDataBuilder()
{
private UserSpec _userSpec = new UserSpec();
public UserTestDataBuilder With(Action&lt;UserSpec&gt; action)
{
action(_userSpec);
return this;
}

public User Build()
{
return new User(_userSpec.Name, _userSpec.Reputation,
_userSpec.ScholarBadge, _userSpec.CriticBadge);
}
}
[/sourcecode]

Inverting the direction of mouse scroll wheel in Windows

Why would you do that?

It's because since in Mac OS X Lion, they've reversed the wheel scroll direction. I've always reversed it back in system preference. Then I've upgraded it to Yosemite and didn't bother to change it. I just change myself to follow it. A few days later, the Mac mouse scroll wheel direction became natural to me.

Now I have a problem with my work PC, which is Windows Server 2012. Whenever I scroll the page with the mouse, the page goes in the opposite direction and it really annoys and confuses me.

So, now you need to reverse the mouse wheel scroll in windows. How can you do that?

You have to edit a registry, unfortunately, but there's a powershell script that does it for you. Of course, you shouldn't run any script you find on Internet in your admin elevated powershell shell. But I took the risk and did it. This is the script.

Get-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Enum\HID\*\*\Device` Parameters FlipFlopWheel -EA 0 | ForEach-Object { Set-ItemProperty $_.PSPath FlipFlopWheel 1 }

Good luck and enjoy the inverted direction of the mouse scroll wheel.

Reactive Extensions, to write async, event-based programs with observables

Hello World in Reactive way

Reactive Programming is "a programming paradigm oriented around data flows and the propagation of change" (wikipedia)

With Reactive Extensions (Rx), you can write asynchronous and event-based programs using observable sequences. Rx let you represent asynchronous data streams with Observables, (push-based notifications) and query asynchronous data streams using LINQ, Simply put "Rx = Observables + LINQ + Schedulers".

You can install the package via nuget.

pm> Install-Package Rx-Main

Channel9 has a concise introduction video: Rx Workshop Introduction. The simplest "Hello, World" can be done in this way.

class Program
{
    static void Main(string[] args)
    {
        var streamOfChars = "Hello, World".ToObservable();
        streamOfChars.Subscribe(c => Console.WriteLine(c));
    }
}

Another simple example is to enumerate from 1 to 10 and subscribe to it.

IObservable<int> source = Observable.Range(1, 10);
IDisposable subscription = source.Subscribe(
   x => Console.WriteLine("OnNext: {0}", x),
   ex => Console.WriteLine("OnError: {0}", ex.Message),
   () => Console.WriteLine("OnCompleted"));


    
    
  

git cherry-pick

Git commit's id is a hash of its contents and its history, and becomes a unique id for a specific commit. Even if it contains the same change, as the parent would be different, it'll have a different id.

"git cherry-pick" takes a commit from somewhere else, and "play it back" where you are right now. Git will build a new commit with a different hash, as the parents are different, though the contents are the same. One thing to note is that git hash is branch agnostic. in Git, an branch is simply "a lightweight movable pointer to one of these commits" (from git branching)

The other day, I rebuilt a release branch for release (of course!). I had to fix one issue, so committed the fix to the release branch. I made a few other changes and revoked the change, as they were not really necessary. Now I wanted to cheery-pick the commit for the fix.

I did git log.

C:\Users\andrew.chaa\Documents\Projects\PopOpen [release]> git log
commit 36bfde24c821f36f84c6ec88c796ae6edac17286
Author: andrewchaa <>
Date: Wed May 13 15:45:34 2015 +0100
the version is updated to 0.8.6


    
    
  

HATEOAS RESTful service

Sample placeholder post.

이민, 새로운 기회와 도전

"당신의 미국 이민이 망하는 다섯 가지 이유" 제목의 슬로우 뉴스 기사를 트위터를 통해 접하고는 잠깐 "광분" 했었다. 그럴꺼 까지는 없었는데. 문든 옛 생각이 났다. 대학원 가기 전, 준비를 위해 석사 1년차이던 선배에게 조언을 구했던 일이. 선배의 답변에 너무 놀랬었다. "야 넌 이길로 오지 마라. 사람이 할 짓이 아니다" 왜 그 선배는, 적어도 내가 보기에는, 석사 과정을 잘 하고 있었으면서도, 후배의 기를 팍 죽이는 말들을 내뱉었던걸까? 그후, 사회생활을 통해, 여러 부류의 사람들을 만나면서 좀 더 이해하게 되었다. 걔 중에는 자신이 가고 있는 길이 얼마나 힘들고 어려운지를 강조함으로써 자신을 높이려는 사람들이 있다는 것을.

좀 흥분이 가라앉은 후, 곰곰히 생각해 봤다. 왜 내가 저 글을 싫어하는지.

첫째는 그 선배 기억이 나서였고, 둘째, 내 경우, 이민을 통해 망하기보다 흥한 경우인데, 저 글이 이를 부정하고 있기 때문이었다.

외국 이민은 국내에서 접할 수 없는 새로운 기회와 경험을 가져다 준다. 전에 Barclays의 Pricing team에서 다른 Quant 들과 일할 때 였다. 나야 Quant팀에 Quant가 아니라 개발자로, 뒷문으로 들어간 격이기 때문에 별 생각이 없었는데, 한국에 방문했을때, 내 직장을 얘기하면 갑자기 나를 쳐다보는 사람들의 눈빛이 달라지는 것을 느꼈다. 그 유명한 Barclays라니. 그 것도 Quant 팀이라니 등등.

런던에 개발자로 일하면서 이곳의 금융기관에서 일하는게 그렇게 어려운 일은 아니다. HSBC도 있고 Credit Suisse도 있고, Barclays, Santander, 등등. 하지만, 만약 내가 한국에 있덨다면, 한국의 개발자 였다면... 정말 실력있는 개발자들 한국에 많지만, 내가 그만큼의 실력이 없어도, 단지 런던에서 개발자로 경력을 쌓아가고 있다는 이유만으로 그런 곳에서 일할 수 있는 기회가 주어진다.

어제 Sam Newman이 Microservice에 얘기하는 강연에 갔다왔다. 궁굼한 것 질문도 해볼 수 있었고.

Embedded image permalink

내가 학부시절 공부했던 Programming Windows with MFC, 2nd edition을 썼던 Jeff Prosise도 런던에서 만났다. 함께 사진도 찍었다. Uni시절에 당신 책으로 공부했었다면서.

이런 기회는 내가 열심히 하냐 안하냐의 노력으로 주어지는게 아니라, 그냥 런던에 살기 때문에 주어지는 기회이다. Sam Newman이 한국가서 강연할 기회가 얼마나 있겠나.

런던의 개발자들, 한국의 개발자에 비하면 2배 이상 받는 느낌이다. 물론 물가도 두배정도 비싸지만. 런던의 버스 기사가 인도의 버스 기사랑 역랑이 비슷하더라도 월급은 몇 배이상 받는 것처럼, 단순히 런던에서 일하기 때문에, 나 역시 나보다 훨씬 실력있는 한국의 개발자들 보다 더 많은 월급을 받고 있다. 6시면 퇴근이고, 요즘 2주간은 아이들 학교 데려다 주느라 10시까지 출근한다. 이 역시 나의 노력여부에 따른 보상이 아니라, 내가 그냥 런던에서 일하기 때문이다.

사람들이 모이는 곳에 기회가 있다. 그래서 한국에 살면 다들 서울로 가려고 하는 것처럼, 세계를 대상으로 보면, 그 가운데, 마치 세계의 수도처럼 사람들이 모이는 곳이 있다. New York이라든지, 개발자라면 실리콘 밸리라든지. 그런곳에 가면, 물론 어려운 점들도 있지만, 가지 않았을 경우, 꿈도 꾸기 어려운 기회들이 너무나도 쉽게 접하게 된다.

그런점에서 난 이민을 적극 추천한다. 특히 한국처럼 기회가 없는 사회에서 기회에 목말라 하며, 저임금, 불안한 일자리에 매여 있는 청년들에게 외국 생활은 새로운 돌파구가 될 수 있다.

"당신의 미국 이민이 망하는 다섯가지 이유" 내게는 "당신이 서울로 이사가면 망하는 다섯가지 이유"처럼 들린다.

NuGet restore

NuGet is a package manager like NPM for node.js or Gem for Ruby, and it is a dominant one in .NET.

You can handle dependency packages in two ways. You download them from NuGet server but also store them in your source control. Or, you just remember what packages you use in your project, and restore them on build. Node and Ruby guys always go for the 2nd option. .NET people used to do the first one, but now prefer the 2nd option. I agree to the 2nd option, restoring packages on build, rather than storing them in git repository. Git is for source code, NuGet repository is for packages.

The Visual Studio on my work machine isn't set up for NuGet restore. "Restore" option is disabled, and when I download and build an open-source project, it can't restore the packages and can't build the solution.

I've checked if there's work-around. It seems that the latest NuGet.exe can restore packages in command-line, regardless of your VS settings, which makes sense to me, as NuGet shouldn't depend on the settings of your IDE. So update NuGet.exe to the latest.

[sourcecode language="powershell"]
exec { .\Tools\NuGet\NuGet.exe update -self }
[/sourcecode]

Create an environmental variable, EnableNuGetPackageRestore, to true, but just for Process, not for Machine, not to impact other projects.

[sourcecode language="powershell"]
[Environment]::SetEnvironmentVariable("EnableNuGetPackageRestore", "true", "Process")
[/sourcecode]

Then, you can safely restore your packages in command-line.

[sourcecode language="powershell"]
exec { .\Tools\NuGet\NuGet.exe restore ".\Src\$name.sln" | Out-Default } "Error restoring $name"
[/sourcecode]

So, all the code in one place

[sourcecode language="powershell"]
Write-Host "Restoring"
[Environment]::SetEnvironmentVariable("EnableNuGetPackageRestore", "true", "Process")
exec { .\Tools\NuGet\NuGet.exe update -self }
exec { .\Tools\NuGet\NuGet.exe restore ".\Src\$name.sln" | Out-Default } "Error restoring $name"
[/sourcecode]