Friday, December 12, 2014

Attribute Caching

Let's talk about caching in C# and making our life easier around it. If you use caching, the most common pattern would be something like:
object GetValue()
{
  object o = GetFromCache("myKey");
  if (o== null)
  {
    o = CalcValue();
    SaveToCache ("myKey", o);
  }
  return o;
}

Pretty simple and obvious, I'm sure you've done it many times. But if you have dozens or hundreds functions like the one above, your code becomes dirty and hard to maintain. You also need to maintain all those unique keys for your cache storage. Wouldn't it be nice if we can write something like:
[Cacheable]
object GetValue()
{
  return CalcValue();
}

Now you can do it exactly like this using AttributeCaching library.

How it Works

CacheableAttribute is translated to C# code behind the scene, which checks if the value is already in cache prior to execute the function. If the value is in cache already then it is returned immediately, and the function body is not even executed. If the value is not in cache yet, then the function is executed, and the value it returns is immediately put into cache. I.e. this is implementation of read-through caching.
As a key to store and read values from cache, the attribute uses a combination of assembly name + method name + passing parameters values. So if you call a function with parameters, a different caching key will be used every time.

Let's consider more complex examples.

Cache Lifetime

If you need your cached values to expire eventually, you can specify different time spans for caching:
[Cacheable (Seconds = 10)]
int CalcEvery10Seconds (int a, int b)
{
  return a+b;
}

[Cacheable (Minutes = 20)]
int CalcEvery20Minutes (int a, int b)
{
  return a+b;
}

[Cacheable (Hours = 1)]
int CalcHourly (int a, int b)
{
  return a+b;
}

[Cacheable (Days = 7)]
int CalcWeekly (int a, int b)
{
  return a+b;
}


Cache Dependencies

It often happens that you need to invalidate cached value immediately, before its time span expires. For example, you save something to database and you want that the cacheable function, which reads the database, returns new values and not the cached ones. Basically you want to purge your cache and ideally just the part required for the function.
To solve this issue you can use dependency tags:
[Cacheable("cars")]
public Car[] GetCars() { ... }

[Cacheable("cars")]
public Car GetCarById (int id) { ... }
Here we tell that GetCars and GetCarById functions depend on "cars" tag.
[EvictCache("cars")]
public void UpdateCar (Car car) { ... }

This example introduces a new attribute EvictCache. When you call UpdateCar() function, it invalidates all cached values with "cars" tag. I.e. after this function is done, the GetCars and GetCarById functions don't have anything in the cache.

More Complex Dependencies

You can also specify multiple dependency tags for a function, and also evict them all all separately. Example:
[Cacheable("cars")]
Car[] GetCars() { ... }

[Cacheable("cars", "cars_jp")]
Car[] GetJapaneseCars() { ... }

[EvictCache("cars", "cars_jp")]
void UpdateCar (Car car) { ... }

In this case UpdateCar will evict everything which has "cars" OR "cars_jp" tags, i.e. cache will be evicted for both GetCars and GetJapaneseCars.

Ignoring Arguments

If cacheable value doesn't depend on all arguments of a cacheable function, you can use CacheIgnore attribute.
[Cacheable]
int Calc(int arg1, [CacheIgnore] int arg2, int arg3) { ... }

Cache Context

Predefined caching rules are really useful, but sometimes we need to break rules. What if you need to change caching duration in runtime? You can access current caching context and manage some rules for that:
[Cacheable (Minutes = 10)]
int Calc(int i)
{
  CacheScope.CurrentContext.LifeSpan = TimeSpan.FromMinutes (1);
  ...
}
In the example you dynamically change cache lifetime from predefined 10 minutes to 1 minute.

Locking

If your cacheable functions is called from several parallel threads at the same time, and it is not cached yet, then you will end up with multiple threads doing the same operation and wasting precious resources. It's a pretty common situation for backend applications: once cache is expired for a heavily using function, avalanche of threads would stuck in that function.
AttributeCaching library prevents parallel execution by default and locks other threads until the already working one is done with that function. I.e. if Thread1 calls F() and shortly after that Thread2 calls F(), Thread2 will be blocked until F() is done in Thread1 and in Thread2 it will be returned the already cached value. What is important is that locking will be done only in the case the function is called with exactly the same arguments.

If for some reason you want to prevent default locking mechanism, use AllowConcurrentCalls property:
[Cacheable (AllowConcurrentCalls = true)]
string Calc(string prop) { ... }

Cache Storage

Some words on what is used to store all caching values. The default storage is MemoryCache from .NET Framework v4. But it can be changed to use Redis as the storage:
//...
CacheFactory.Cache = new RedisCacheAdapter (RedisConnectionString);
//...


Wednesday, October 22, 2014

Multiple RoutePrefix

In MVC5 Microsoft has added routing using C# attributes. It is hard to overestimate this feature: finally you don't need to switch between your controllers and RoutesConfig.cs to find out where all your endpoints go.
To make it even sweeter they added RoutePrefix attribute, which sets one prefix to multiple actions. Though you can set multiple Route attributes for each action, you cannot set multiple RoutePrefix. And this is very frustrating when you need to have multiple paths to the same action. Luckily, the framework is very customizable (many thanks for that), so we can easily fix this drawback.

Introducing RouteMPrefix attribute, which supports multiple instances and your code looks like:
[RouteMPrefix("v1/values")]
[RouteMPrefix("v2/values")]
public class ValuesController : ApiController
{
  [Route("list")]
  [Route("get")]
  public IEnumerable Get()
  {
    return new[] { "value1", "value2" };
  }
}

As a result all the following routes will be created:
  • /v1/values/list
  • /v1/values/get
  • /v2/values/list
  • /v2/values/get

And to make it all work you need:
1. Add two files MultiPrefixRouteProvider.cs and RouteMPrefixAttribute.cs from this GitHub repository

2. Use MultiPrefixRouteProvider when initializing attribute routing:
config.MapHttpAttributeRoutes (new MultiPrefixRouteProvider());


This is it. Happy coding.

Monday, January 20, 2014

Embedding GIT Commit Hash to Assembly Version

Have you ever tried to understand looking at your DLL what GIT branch it corresponds to? Have you been thinking if your commit is already in this DLL or not?

To easily find a GIT commit which your binary assembly relates to, it would be very handy to have the GIT commit hash in the assembly info. And having something like "1.0.0.0-e7f45a6" instead of just "1.0.0.0" in you file properties:



There are already several solutions to do that automatically (you were not planning to maintain it manually, right?) in the Internet, but they have some drawbacks. Automatic update of AssemblyInfo.cs with every build will create a real nightmare in commits log. Using GIT hooks is not bad but you need to have hooks on every developer's machine. So ideally, we need something that would dynamically modify assembly version on each build.

This solution appends commit hash to your assembly's version and puts it all together to AssemblyInformationalVersion attribute, so in your assembly file properties you would have something like "Product version: 1.0.0.0-e7f45a6".

Usage

1. Add MSBuildTasks using Nuget to your project(s). Anything from the version 1.4.0.65 would work.
2. Create file GitAssemblyVersion.targets in $(SolutionDir)\.build folder with content:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <MSBuildCommunityTasksPath Condition=" '$(MSBuildCommunityTasksPath)' == '' ">$(SolutionDir)\.build</MSBuildCommunityTasksPath>
    <AssemblyInfoFile Condition=" '$(AssemblyInfoFile)' == '' ">$(MsBuildProjectDirectory)\Properties\AssemblyInfo.cs</AssemblyInfoFile>
    <GeneratedAssemblyInfoFile Condition=" '$(GeneratedAssemblyInfoFile)' == '' ">$(MsBuildProjectDirectory)\Properties\GeneratedAssemblyInfo.cs</GeneratedAssemblyInfoFile>
    <BuildDependsOn>
      ReadAssemblyVersion;
      SetAssemblyVersion;
      $(BuildDependsOn)
    </BuildDependsOn>
    <CleanDependsOn>
      $(CleanDependsOn);
      SetAssemblyVersionClean
    </CleanDependsOn>
  </PropertyGroup>
  
  <Import Project="$(MSBuildCommunityTasksPath)\MSBuild.Community.Tasks.targets" />
  
  <Target Name="ReadAssemblyVersion">
    <ReadLinesFromFile File="$(AssemblyInfoFile)">
      <Output TaskParameter="Lines" ItemName="ItemsFromFile"/>
    </ReadLinesFromFile>
    <PropertyGroup>
      <Pattern>;\s*\[assembly\s*:\s*AssemblyVersion.*?"(.*?)"</Pattern>
      <In>@(ItemsFromFile)</In>
      <AssemblyVersion>$([System.Text.RegularExpressions.Regex]::Match($(In), $(Pattern)).Groups.get_Item(1))</AssemblyVersion>
    </PropertyGroup>
  </Target>
  <Target Name="SetAssemblyVersion">
    <ItemGroup>
      <Compile Include="$(GeneratedAssemblyInfoFile)" />
    </ItemGroup>
    <GitVersion LocalPath="$(SolutionDir)">
      <Output TaskParameter="CommitHash" PropertyName="CommitHash" />
    </GitVersion>
    <AssemblyInfo CodeLanguage="CS" OutputFile="$(GeneratedAssemblyInfoFile)" AssemblyInformationalVersion="$(AssemblyVersion)-$(CommitHash)" />
  </Target>
  <Target Name="SetAssemblyVersionClean" Condition="Exists($(GeneratedAssemblyInfoFile))">
    <Delete Files="$(GeneratedAssemblyInfoFile)" />
  </Target>
</Project>

3. Add this line somewhere to the end of you .csproj file(s):
<Import Project="$(SolutionDir)\.build\GitAssemblyVersion.targets" />
That's it!

How It Works

It dynamically creates GeneratedAssemblyInfo.cs file, extracts version from AssemblyVersion attribute, and adds current GIT commit hash before each build.

Tuesday, November 26, 2013

Avoiding Duplicate Query Parameters in Web API

If you pass parameters to your Web API using queries, you would have a method like:
public IEnumerable<string> Get([FromUri] string p1)...
And you'll call it like:
.../Values?p1=a
All of this works perfectly fine until someone passes you duplicate parameters:
.../Values?p1=a&p1=b
In this case your Web API method, instead of getting the first value "a" or the last one "b", will have "(Collection)". Of course, you can redefine your method and make it to accept arrays, then validate the arrays, check if they are not empty, have mo than one element, and so on. But if you need only one unique value and ignore all the rest ones, Web API provides you with a nice but not very obvious solution. To do this you need to redefine the way the Web API parses query parameters. And the responsible ones for this functionality are the ValueProvider classes. There are different providers, but we need QueryStringValueProvider, and we simply replace it with our own implementation. 1. Replacing QueryStringValueProvider with our implementation (should be done anywhere in WebApiConfig):
config.Services.Remove (typeof (ValueProviderFactory), config.Services.GetValueProviderFactories().First (f => f is QueryStringValueProviderFactory));
config.Services.Add (typeof(ValueProviderFactory), new QueryStringUniqueValueProviderFactory());
2. Our implementation of QueryStringValueProvider providing only unique values:
using System.Globalization;
using System.Collections.Generic;
using System.Net.Http;
using System.Web.Http.Controllers;
using System.Web.Http.ValueProviders;
using System.Web.Http.ValueProviders.Providers;

public class QueryStringUniqueValueProviderFactory : QueryStringValueProviderFactory
{
 private const string RequestLocalStorageKey = "{755EDBD6-46CD-4B44-8162-31D8CF111155}";

 public override IValueProvider GetValueProvider(HttpActionContext actionContext)
 {
  object provider;
  var storage = actionContext.Request.Properties;

  // Only parse the query string once-per request
  if (!storage.TryGetValue(RequestLocalStorageKey, out provider))
  {
   provider = new QueryStringUniqueValueProvider(actionContext, CultureInfo.InvariantCulture);
   storage[RequestLocalStorageKey] = provider;
  }

  return (IValueProvider)provider;
 }
}


public class QueryStringUniqueValueProvider : NameValuePairsValueProvider
{
 public QueryStringUniqueValueProvider(HttpActionContext actionContext, CultureInfo culture)
  : base(GetUniqueQueryNameValuePairs(actionContext), culture)
 {
 }


 private static IEnumerable<KeyValuePair<string, string>> GetUniqueQueryNameValuePairs(HttpActionContext actionContext)
 {
  var pairs= actionContext.ControllerContext.Request.GetQueryNameValuePairs();
  var returnedKeys= new HashSet<string>();
  foreach (var pair in pairs)
  {
   if (returnedKeys.Contains (pair.Key))
    continue;
   returnedKeys.Add (pair.Key);
   yield return pair;
  }
 }
}
And that's it! One place, no mess in your code with arrays validation, quick and neat.

Sunday, June 23, 2013

Connecting Raspberry Pi and Arduino. Software

In my previous post I described how to make hardware connection between Raspberry Pi and Arduino. Now I will describe the software part. I will describe how to make software connection from scratch pretty easily. And I will use some C and C#.

Raspberry Pi

I use Arch Linux on my Raspberry Pi (more lightweight and appropriate distributive for this computer to my mind). But I2C drivers work the same way on other distributives, so, only some system specific commands could be different.

1. Update your system. I2C drivers were released in Fall 2012, so if your system is old enough do this
$ pacman -Syu

2. Load I2C kernel module:
$ modprobe i2c-dev
To load the module automatically on boot, create file /etc/modules-load.d/i2c.conf and add line i2c-dev to the file.

3. Check I2C works. Run
$ i2cdetect 1
Where 1 is the number of I2C bus. The command should output table with addressed of all connected devices. But for now that table is empty as far as we haven't initialized the Arduino as an I2C slave.

Arduino

To work with I2C on Arduino you can use the standard library Wire. The code below initializes the I2C bus, reads sending data and sends data back.
#include <Wire.h>

void setup()
{
  Serial.begin(115200);          // start serial for output
  Wire.begin(42);                // join i2c bus with address #42
  Wire.onReceive(receiveData);
  Wire.onRequest(sendData); 
}

void loop()
{
  delay(100);
}

// callback for received data
void receiveData(int byteCount) 
{
  while(Wire.available())       // loop through all but the last
  {
    char c = Wire.read();       // receive byte as a character
    Serial.print(c);            // print the character
  }
}

// callback for sending data
void sendData()
{ 
  Wire.write (79);
}
After uploading this code to Arduino we can get back to the Raspberry Pi.

Raspberry Pi again

Now, after initializing the Arduino, i2cdetect command should show you that a device with address 42 is connected to the I2C bus. If that works, it's time to send some data from Raspberry Pi. I'm a dotNET guy, and I will use C# and Mono. And to work with I2C I use my library RPi.I2C.Net. The code below is the simple application to send and read data from the Arduino.
static void Main(string[] args)
{
 using (var bus = RPi.I2C.Net.I2CBus.Open("/dev/i2c-1"))
 {
  bus.WriteBytes(42, new byte[] { 49, 50, 51 });
  byte[] res = bus.ReadBytes(42, 1);
  Console.WriteLine(res[0]);
 }
}
Now it's time to create more complex communication.

Saturday, June 22, 2013

Connecting Raspberry Pi and Arduino. Hardware

Though you can control peripheral hardware using Raspberry Pi's GPIO (general ports for input/output), I prefer to do that using an array of Arduino. The advantages are:
  • No limitation in ports. You can always use Arduino Mega, or even use an array of them.
  • Physical separation of controller and Arduino.
  • It's not that scary to burn another Arduino for $10 rather than Raspbery Pi for $35.
And the most effective way to connect several Arduino and Raspberry Pi is I2C (or I2C to be precise). You need only 3 wires (4 in practice) to connect even several Arduino.

So the general connection schema using I2C looks like this:


Unfortunately, you cannot insert wires from Arduino to Raspberry Pi. The problem is the I2C bus does not strictly specify the signal levels. As a result we have 5V for Arduino's I2C implementation, and 3.3V for Raspberry Pi's one. And if you connect the I2C pins directly, you're in great danger of burning your Raspberry Pi (or at least part of it). So we have to use an I2C bridge, or level shifter. You can buy a ready one or create you own, as I did. And the schema for the bridge is pretty simple:

It requires 4 N-channel MOSFETs (I used BS170) and 4 pull-up resistors (10K each). Another important moment is Arduino and Raspberry Pi need to have the common ground. So simply join the ground pins (and that would the the fourth required wire I was talking about earlier).

After soldering I got this:


And the common connection scheme using the I2C bridge now looks like:


Scheme of pins for Raspberry Pi can be found here. And for Arduino SDA is A4, SCL is A5.

That's it with the hardware part. In this post I describe the software part of connecting Arduino and Raspberry Pi.