https://www.henrik.org/

Blog

Thursday, June 25, 2015

Designing for failure

One of the first things you hear when you learn about how to design for the cloud is that you should always design for failure. This generally means that any given piece of your cloud infrastructure can stop working at a given time so you need to design for this when constructing your architecture and gain reliability by creating your application with redundancy so that any given part of your applications infrastructure can fail without affecting the actual functionality of the website.

Here is where it gets tricky though. Before I actually started running things in a cloud environment I assumed this meant that every once in a while a certain part of your infrastructure (For instance a VM) would go away and be replaced by another computer within a short time. That is not what designing for failure means. To be sure this happens too, but if that was the only problem you would encounter you could even design your application to deal with failures in a manual way once they happen. In my experience even in a relatively small cloud environment you should expect random intermittent failures to happen at least once every few hours and you really have to design every single piece of your code to handle failures automatically and work around them.

Every non local service you use, even the once that are designed for ultra high reliability like Amazon S3 and Azure Blob Storage can be assumed to fail a couple of times a day if you make a lot of calls to them. Same thing with any database access or any other API.

So what are you supposed to do about it. The key thing is that whenever you try to do anything with a remote service you need to verify that the call succeeded and if it didn't keep retrying. Most failures that I have encountered are transient and tend to pass within a minute or so at the most. The key is to design your application to be loosely coupled and whenever a piece of the infrastructure experiences a hiccup you just keep retrying it for a while and usually the issue will go away.

Microsoft has some code that will help you do this as well which is called The Transient Fault Handling Block. If you are using the Entity Framework everything is done for you and you just have to specify a Retry Execution Strategy by creating a class like this.

    public class YourConfiguration : DbConfiguration 
    { 
      public YourConfiguration() 
      { 
        SetExecutionStrategy("System.Data.SqlClient",
                             () => new SqlAzureExecutionStrategy()); 
      } 
    }

Then all you have to do is add an attribute specifying to use the configuration on your Entity context class like so.

    [DbConfigurationType(typeof(YourDbConfiguration))] 
    public class YourContextContext : DbContext 
    { 
    }

It also comes with more generic code for retrying execution. However I am not really happy with the interface of the retry policy functionality. Specifically, there is no way that I could figure out to create a generic log function that allows me to log the failures where I can see what is actually requiring retries. I also don't want to have a gigantic log file just because for a while every SQL call takes 20 retries each one being logged. I rather get one log message per call that indicates how many retries were required before it succeeded (Or not).

So to that effect I created this little library. It is compatible with the transient block mentioned earlier in that you can reuse retry strategies and transient exception detection from this library. It does improve on logging though as mentioned before. Here is some sample usage.

      RetryExecutor executor = new RetryExecutor();
      executor.ExecuteAction(() =>
        { ... Do something ... });
      var val executor.ExecuteAction(() =>
        { ... Do something ...; return val; });
      await executor.ExecuteAsync(async () =>
        { ... Do something async ... });
      var val = await executor.ExecuteAsync(async () =>
        { ... Do something async ...; return val; });

By default only ApplicationExceptions are passed through without retries. Also the retry strategy will try 10 times waiting for the number of previously tries seconds until the next try (Which means it will signal a failure after around 55 seconds). The logging will just write to the standard output.

Saturday, June 20, 2015

Simple Soap envelope parser and generator

So I figured as a followup to my previous post here is a small sample project of what I would love to find when searching for a person online that is looking for a job.

This library on GitHub is just one class to help you generate and parse a SOAP envelope, something I was surprised to see wasn't actually available in the .Net framework as a stand alone class (Or at least I haven't found it).

Its use is very simple. To create a SOAP envelope you create an instance of the class SoapEnvelope and assign the Headers and `Body` properties (And possible the Exception if you want to signal an error) and then call the ToDocument method to generate the XML document for the SOAP envelope.

To read data simply call the method SoapEnvelop.FromStream or SoapEnvelope.FromRequest and it will return the envelope it parsed from the stream or request. It does support handling GZip content encoding from the request.

Here is a simple round trip example of its use (For more examples check out the tests).

      SoapEnvelope envelope = new SoapEnvelope();
      envelope.Body = new[] { new XElement("BodyElement") };
      envelope.Headers = new[] { new XElement("HeaderElement") };
      XDocument doc = envelope.ToDocument();
      MemoryStream stream = new MemoryStream();
      doc.Save(stream);
      stream.Seek(0, SeekOrigin.Begin);
      SoapEnvelope otherEnvelope = SoapEnvelope.FromStream(stream);

To continue from the previous post from a few days ago. Even though this example is very short it does show a couple of things if I were to evaluate the author of something similar for a job.

  • This is somebody who actually likes to code because otherwise why would he (Or she) even have taken the time to do this.
  • This is somebody that cares about the quality of their code because even though this is a super simple class it contains a small test suite to make sure that it works.
  • This person has at least a decent grasp of the C# and .Net framework and understand how to use inheritance and interfaces to create something new (If you are a coder and doesn't know, it is scary how few people who should know this stuff, do actually know it).

Thursday, June 18, 2015

What I look for when evaluating future hires

Even though I am not a manager and have put a high importance of never becoming one as one of my own personal development goals I do quite often chime in on evaluating future hires both currently for permanent positions or in the past for consultancy contracts and there is one thing that it seems to be an important thing that a lot if not even most software developers are not doing that I put a high premium on when evaluating new candidates for job application.

The first thing I do when I get a resume sent to me for a prospective candidate is that I go to Google and search for their name. If I can't find a single program related name from anything they've ever done online that is a pretty big blotch on their record from my perspective.

My thinking for this is that to be good at software development and like solving problems even if you are straight out of school you will have done one of the following.

  • Asked a question you couldn't figure out, or even better provided an answer to a question for somebody else, on a site like Stack Overflow or CodeProject.
  • Create or participated in an open source project hosted on GitHub or SourceForge.
  • Created some weird obscure website somewhere (Doesn't really matter what it is or how much traffic it has).
  • Create a blog about something. It doesn't have to be old or very active, but at least you've tried.
  • Have some sort of presence I can find on social media, preferably with some comments I can find in relation to software development. Doesn't matter if it is Twitter, Facebook, LinkedIn, Google+ or whatever.

The more of these you can check off the better, but if I can't find you at all that is a huge red flag in my book and you would hopefully be surprised at how common this is for would be software developers.

The problem is if you haven't gotten around to anything of the above to me that signifies that you aren't that into software development and it is just something you do, and generally good coders really like to code and they do it because they like it. If I couldn't make a living for coding I would do it anyway, and most of my public presence online is based on the work I've done when I haven't been collecting a paycheck for it (Since most of the work you do when you do get paid you can't just publish online).

So my advice to anybody who wants to get started working with software development is to sign up for a free account on GitHub and just find a small itch and write an open source application to scratch it. And make sure the repository is associated with your real name so that when I or any other person involved in any hiring search for you we will find it. I can almost guarantee that it will be worth your time.

Thursday, May 7, 2015

C# Task scheduling and concurrency

It is very hard to figure out how the new async Task API for handling threading and concurrency works in .Net 4.5. I have dug around a lot to try and find any documentation on this topic and have mostly failed so when in doubt I decided to simply figure it by writing some test applications that checked how it actually behaved. It is important to note that this is how threading works in a console .Net 4.5 application on Windows 8.1. I would not be surprised if specific numbers of the thread model were different in a server setting, different OS version or even .Net versions. So without further ado here are my findings.

First of all if you simply start a lot of Task's that all run for a long time you quickly notice that by default the .Net runtime will allocate a minimum of 8 threads to run tasks. Then it gets interesting though because for every second that the task queue keep being full another thread is added. This keeps going all the way up to a maximum of 1023 threads. After 1023 threads have been allocated no more threads will be allocated for any reason so any remaining tasks will wait to start until a previous task has completed. If a thread executes no tasks at all for 20 seconds it will be removed from the thread pool.

There are also odd things happening with the order of which tasks are scheduled. For instance if you were to run the following code below it will run very slowly because no threads from the second for loop will be scheduled to run until the thread pool has expanded to run all tasks from the first loop concurrently (So for almost 100 seconds no processing will happen).

      for (int i = 0; i < 100; i++)
      {
        int thread = i;
        firstTasks.Add( Task.Run(() =>
        {
          Thread.Sleep(100);
          // Do something else
          secondTasks[thread].Wait();
        }));
      }

      for (int i = 0; i < 100; i++)
      {
        secondTasks.Add( Task.Run(() =>
        {
          // Do something in the background.
        }));
      }

In fact if you increase the upper bound of i from 100 to 1024 this example will never finish since all the 1023 possible available threads will be taken up with this initial tasks waiting for second tasks to finish which will never be scheduled for execution because of thread exhaustion.

This might seem like a contrived example, but it is actually not that uncommon to end up with a similar scenario if you use non async code inside a task in a complicated multithreaded application. If you instead write the code below like this it will complete almost immediately and not have any issues regardless of how many iterations of the loop you make because the second thread when created within the affinity of the thread that then waits for it actually causes the second thread to be executed immediately on that thread (As long as it hasn't been scheduled to run on another thread already).

      for (int i = 0; i < 100; i++)
      {
        Task.Run(() =>
        {
          Thread.Sleep(100);
          Task secondTask = Task.Run(() =>
          {
            // Do something in the background.
          });
          // Do something else
          secondTask.Wait();
        }));
      }

One last thing you have to be very careful about when it comes to task, especially when using the async syntax is that you have to realize that once you await on something there is absolutely no guarantee that once the execution continues it is on the same thread. So for instance this code is just waiting to creating a deadlock that will be really hard to track down.

      object lockObj = new object();

      Monitor.Enter(lockObj);
      await MethodAsync();
      Monitor.Exit(lockObj);

There really is no way to handle locking securely but if you absolutely need to do locking of a resource while doing async coding you could possibly use semaphores which do not require being reset from the same thread. This generally doesn't lead to good code though and generally if you think about where your synchronization code is you can avoid having locks over awaits but it might take a little bit of extra work.

Sunday, January 25, 2015

Choosing your cloud provider

When you start any coding project you generally need some sort of server capability even if the application your building is not a web site. When choosing your cloud provider there are several different things to think about.

First of all you need to consider if what you need is very basic and will not require a high SLA or the ability to grow with usage you are probably better off choosing a virtual private server provider. If you are fine with a Linux box these can be had extremely cheap. I used to be a customer of Letbox and at the time they provided me with a Virtual Private Server for $5/month, a price that is hard to beat. It is however important to realize that this is not a true VM, it is a specialized version of Linux similar to doing a chroot but also with quotas on memory and CPU usage. This means that these VM:s can only run Linux. That said the price is simply in a league of itself, cheaper usually than even spot instances of AWS EC2.

However once you have something slightly more complicated to run you probably want to go with a "real" cloud provider. These come in two kinds. The first level are companies providing infrastructure as a service (IaaS). This means basically providing virtual machines, storage and networking for them. It is up to you to build everything you need to run off of these primitives. Companies that offer only this kind of computing includes Skytap, Rackspace (Although Rackspace does have some platform services) and many more.

The next level up are the companies that provide platform as a service (PaaS). All of these companies also provide the infrastructure as well if you need it, but on top of this they provide useful packages that they will run for you as managed services to help creating, deploying and running your services easier. These minimally usually include.

  • Load balancing of traffic to multiple servers.
  • Auto scaling of new servers to handle varying load.
  • A fast and scalable NoSQL key value store.
  • A managed transactional database.
  • Web hosting.

There are as I see it three main players in this space and they are Amazon Web Services (AWS), Microsoft Azure and Google App Engine.

Of these Amazon is by far the largest. AWS started out as a mainly platform as a service offering, but now has one of the most complete set of managed services and they have by far the largest set of data centers located all around the world and one region qualified for US government work loads (Having an account on it requires you to be a US citizen so I can not use it). Their infrastructure is truly top notch, but their development tools are not great. Only a few languages have an official SDK (I myself have been missing an SDK for PERL).

Microsoft approached this space from the opposite direction from Amazon and started out by offering specific platform solutions and tightly integrating the development and deployment of Azure applications into their development tool Visual Studio. It is the only cloud provider I am aware of that for a time did not provide IaaS at all (Although they do now). The SDK and tooling for all of their products is truly excellent, especially if you are a .Net C# developer, but many other languages are supported as well. They do unfortunately and understandably run most of their infrastructure on Windows which simply is not as solid as other hyper-visors out there and if you are building a solution that requires reliable quick processing this can be a problem, especially if you have a cluster of synchronized machine this can become really problematic. These synchronization issues usually only occur a few times a month though as the service is migrated to new machines as all the VM:s running the service undergo the monthly Windows patching. However as long as your application does not rely on tight synchronization between several systems you are unlikely to notice it.

Finally there is Google. Their solution is similar to Amazon something that has grown out of their own business and they have several offerings that are obviously simply surfacing of their internal operations like for instance Big Query. Google's infrastructure is fantastic in regards to reliability and performance. They do though in my opinion offer the most narrow platform solution of the big three. What they do provide though is truly top notch, and they are also priced accordingly unfortunately.

Price wise the big three are relatively similarly priced. If your application can take advantage of the AWS spot pricing you can get away with really cheap solutions though. Google is usually the most expensive (I say usually since the prices change all the time for cloud services). One thing that could be worth investigating is if you qualify for a Microsoft Bizspark membership because if you do you will receive $150/month of free credits to use for almost anything in Microsoft Azure (And it also includes licenses to almost every product that Microsoft has in their very extensive portfolio).

Saturday, January 24, 2015

How to get more free build or test minutes with your Visual Studio Online account

If you are one of the lucky ones who has an MSDN or a BizSpark subscription (One of the best deals around on the internet) and use the hosted build environment of Visual Studio Online it is tricky that you only get 60 minutes of free build time a month if you want to do continuous integration (Which you should do!) using it. However I just discovered a trick how to get around this limit by accident.
  1. First of all log into your Azure management console and then go to the tab for Visual Studio Online subscriptions.
  2. Then click on the unlink button at the bottom when you have the subscription you want new minutes for building or testing on. You will get a warning about losing any licenses you have purchased through your Azure subscription for Visual Studio Online so if you have done that you can't use this trick.
  3. Then click on new at the bottom left of the management screen to link your Visual Studio Online account back to your Azure subscription.
  4. Select the Visual Studio Online account you unlinked earlier and make sure you have the correct subscription selected in the drop down (It defaults to the pay as you go subscription so you will need to change this).
  5. Press the link button in the lower right.

That's it, if you go back your home page on Visual Studio Online you should be able to see that you have a new allotment of build and test minutes.

DISCLAIMER: You might be violating your terms of service with Microsoft by doing this and I also expect Microsoft to fix this at some point so you at your own risk.

Thursday, January 22, 2015

Problems with singletons

One of the most basic software design patterns is the singleton pattern and you'd think this wouldn't be one that would cause you problems but in C# it can be surprisingly difficult and I just spent a couple of hours trying to track down a bug because I hadn't implemented one properly. The place in question was using the first implementation and was being used from multiple threads that all started at the same time.

This is the simple pattern and it almost always works except when you are doing the first access from multiple threads at the same time, but when it doesn't it can be a real hard bug to find.

  internal class Simple
  {
    private static Simple instance;

    public static Simple Instance
    {
      get
      {
        if (instance == null)
          instance = new Simple();
        return instance;
      }
    }
  }

It should be pretty obvious that this class would have problems with concurrency so the simple solution is to just add a lock around the whole thing.

  internal class Lock
  {
    private static readonly object lockObj = new object();
    private static Lock instance;

    public static Lock Instance
    {
      get
      {
        lock (lockObj)
        {
          if (instance == null)
            instance = new Lock();
        }
        return instance;
      }
    }
  }

This class is simple and does work, but getting the lock has a performance penalty which makes it useful to keep looking.

  internal class DoubleLock
  {
    private static readonly object lockObj = new object();
    private static DoubleLock instance;

    public static DoubleLock Instance
    {
      get
      {
        if (instance == null)
        {
          lock (lockObj)
          {
            if (instance == null)
              instance = new DoubleLock();
          }
        }
        return instance;
      }
    }
  }

This class is a little bit more complicated, but it has the advantage that except for the very first check there is no locking required. It does rely on the assignment of a reference to a variable being an atomic assignment, but this is fortunately a valid assumption.

However, you can also use the C# runtime to help you create the singleton using a static constructor.

  internal class Static
  {
    private static Static instance = new Static();

    public static Static Instance
    {
      get
      {
        return instance;
      }
    }
  }

This is pretty much as efficient as it gets, you even got rid of the check for null. And this is thread safe as well. It does have the disadvantage of the instance of the singleton being created right before the first access of anything in the class, which might not be what you are looking for if there are more static methods in the class. The following class is based on the previous concept but does not create the singleton until the first time it is accessed.

  internal class DoubleLazy
  {
    private static class LazyLoader
    {
      public static DoubleLazy instance = new DoubleLazy();
    }

    public static DoubleLazy Instance
    {
      get
      {
        return LazyLoader.instance;
      }
    }
  }

The nested class static constructor will not be called until you read the instance. If you are running C# 4.0 or later there is a helper class that makes this easy to do as well using a lambda expression.

  internal class NewLazy
  {
    private static Lazy instance = new Lazy(() => new NewLazy());

    public static NewLazy Instance
    {
      get
      {
        return instance.Value;
      }
    }
  }

This method also allows you to check if you have instantiated the singleton or not (You can still do that with the first implementations, but it is not possible with any of the ones that using the static initializer). So which one should you chose. It might depend on different aspects, but if the only thing you care about is performance I made some relatively unscientific measuring and came up to the following list.

  • The simple static initializer is the absolute fastest implementation.
  • The nested static initializer is only slightly slower.
  • The simple non thread safe solution is slightly slower.
  • The double lock solution is only slightly slower than the the previous three.
  • The lazy lambda expression solutioin takes roughly 50% longer to run than any of the previous solutions.
  • The lock solution is roughly 150% slower than any of the first 4 solutions.

That said even the slowest solution can still perform roughly 40 million accesses to the singleton per second from a single thread on my laptop so unless you access it a lot it really doesn't matter.