Tasks with Timeouts – Contd.

As mentioned in the last post we can now have tasks with individual timeouts. The code looked a little heavy. Can we do better ? Yes we can !!


var tasks = new List<Task>();
try
{
//instead of checking for fault within the continutaion, 
//we can just use a TaskContinuationOption to tell communicate the right semantics

var t1 = Task.Factory.StartNew(_ => LongRunningTask(), TaskCreationOptions.AttachedToParent)
              .TimeoutAfter(1000)
              .ContinueWith(t => SomethingUsefulWithTheResult(), 
                                 TaskContinuationOptions.NotOnFaulted);

var t2 = Task.Factory.StartNew(_ => LongRunningTask(), TaskCreationOptions.AttachedToParent)
              .ContinueWith(t => SomethingUsefulWithTheResult());

var t3 = Task.Factory.StartNew(_ => LongRunningTask("Entering task3"), TaskCreationOptions.AttachedToParent)
              .ContinueWith(t => SomethingUsefulWithTheResult());
            
tasks.Add(t1);
tasks.Add(t2);
tasks.Add(t3);
                            
Task.WaitAll(tasks.ToArray());
}
catch (Exception ex)
{                
    Console.WriteLine("There was an exception");
    Console.WriteLine(ex.InnerException.Message);               
}
Advertisements

Tasks with Timeouts

So the task is to timeout a task. now, I never thought it would take me as long as it did. Turns out it is a really tricky problem problem. I was expecting something within the framework to make life easier, there isn’t anything by default but msdn to the rescue Tasks With Timeout.

So, the extension method is

    public static class TaskWithTimeout
    {
        internal struct VoidTypeStruct
        { }
        internal static void MarshalTaskResults<TResult>(Task source, TaskCompletionSource<TResult> proxy)
        {
            switch (source.Status)
            {
                case TaskStatus.Faulted:
                    proxy.TrySetException(source.Exception);
                    break;
                case TaskStatus.Canceled:
                    proxy.TrySetCanceled();
                    break;
                case TaskStatus.RanToCompletion:
                    Task<TResult> castedSource = source as Task<TResult>;
                    proxy.TrySetResult(
                        castedSource == null ? default(TResult) : // source is a Task
                            castedSource.Result); // source is a Task<TResult>
                    break;
            }
        }
        
        public static Task TimeoutAfter(this Task task, int millisecondsTimeout)
        {
            // Short-circuit #1: infinite timeout or task already completed
            if (task.IsCompleted || (millisecondsTimeout == Timeout.Infinite))
            {
                // Either the task has already completed or timeout will never occur.
                // No proxy necessary.
                return task;
            }

            // tcs.Task will be returned as a proxy to the caller
            TaskCompletionSource<VoidTypeStruct> tcs = new TaskCompletionSource<VoidTypeStruct>();

            // Short-circuit #2: zero timeout
            if (millisecondsTimeout == 0)
            {
                // We've already timed out.
                tcs.SetException(new TimeoutException());
                return tcs.Task;
            }

            // Set up a timer to complete after the specified timeout period
            Timer timer = new Timer(state =>
            {
                // Recover your state information
                var myTcs = (TaskCompletionSource<VoidTypeStruct>)state;
                // Fault our proxy with a TimeoutException
                myTcs.TrySetException(new TimeoutException());
            }, tcs, millisecondsTimeout, Timeout.Infinite);

            // Wire up the logic for what happens when source task completes
            task.ContinueWith(antecedent =>
                                {
                                    timer.Dispose(); // Cancel the timer
                                    MarshalTaskResults(antecedent, tcs); // Marshal results to proxy
                                },
                                CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);

            return tcs.Task;
        }

        public static Task<TResult> TimeoutAfter<TResult>(this Task<TResult> task, int millisecondsTimeout)
        {
            // Short-circuit #1: infinite timeout or task already completed
            if (task.IsCompleted || (millisecondsTimeout == Timeout.Infinite))
            {
                // Either the task has already completed or timeout will never occur.
                // No proxy necessary.
                return task;
            }

            // tcs.Task will be returned as a proxy to the caller
            TaskCompletionSource<TResult> tcs = new TaskCompletionSource<TResult>();

            // Short-circuit #2: zero timeout
            if (millisecondsTimeout == 0)
            {
                // We've already timed out.
                tcs.SetException(new TimeoutException());
                return tcs.Task;
            }

            // Set up a timer to complete after the specified timeout period
            Timer timer = new Timer(state =>
                                    {
                                        // Recover your state information
                                        var myTcs = (TaskCompletionSource<TResult>)state;
                                        // Fault our proxy with a TimeoutException
                                        myTcs.TrySetException(new TimeoutException());
                                    }, tcs, millisecondsTimeout, Timeout.Infinite);

            // Wire up the logic for what happens when source task completes
            task.ContinueWith(antecedent =>
                                {
                                    timer.Dispose(); // Cancel the timer
                                    MarshalTaskResults(antecedent, tcs); // Marshal results to proxy
                                }, 
                                CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously,TaskScheduler.Default);

            return tcs.Task;
        }
    }

A lot of code for doing this, and the msdn article remains the better source of explanation.

Now using this.

public class Program
	{
		private static List<int> Output = new List<int>();
		private static Random _random = new Random();
		
		public static int LongRunningTask(string message)
		{
			Console.WriteLine(message);
			Console.WriteLine("Managed thread Id " + Thread.CurrentThread.ManagedThreadId);
			//Simulate a long running task
            Thread.Sleep(TimeSpan.FromSeconds(2));
			var number = _random.Next();
			Console.WriteLine("Adding " + number + " From thread  - " + Thread.CurrentThread.ManagedThreadId);
			return number;
		}

		public static void Main(string[] args)
		{
			Console.WriteLine("In Main");
			Console.WriteLine("Managed thread Id " + Thread.CurrentThread.ManagedThreadId);
			var cts = new CancellationTokenSource();
			var tasks = new List<Task>();
			try
			{
//In the continuation check for the condition of fault (or something more if you so need) and perform the //continuation
				var t1 = Task.Factory.StartNew(_ =>	LongRunningTask("Entering task1"),
												    TaskCreationOptions.AttachedToParent)
                                     .TimeoutAfter(1000)
                                     .ContinueWith(antecedent => {
						if(!(antecedent.IsCanceled || antecedent.IsFaulted))
                                                         Output.Add(antecedent.Result);
								}
								, cts.Token);
				var t2 = Task.Factory.StartNew(_ => LongRunningTask("Entering task2"),
													TaskCreationOptions.AttachedToParent)
                                     .ContinueWith(_ => Output.Add(_.Result));
                var t3 = Task.Factory.StartNew(_ => LongRunningTask("Entering task3"), 
													TaskCreationOptions.AttachedToParent)
                                     .ContinueWith(_ => Output.Add(_.Result));
            
                tasks.Add(t1);
                tasks.Add(t2);
                tasks.Add(t3);
                            
                Task.WaitAll(tasks.ToArray());
            }
            catch (Exception ex)
            {                
                Console.WriteLine("There was an exception");
                Console.WriteLine(ex.InnerException.Message);               
            }

            Console.WriteLine("Output :");
            Output.ForEach(_ => Console.WriteLine(_));

            Console.ReadLine();
        }
    }

The important part being that the continuation is applied after the timeout and it won’t work with the other way around.
The output therefore basically looks like this :

In Main
Managed thread Id 9
Entering task1
Managed thread Id 10
Entering task2
Managed thread Id 11
Entering task3
Managed thread Id 14
Adding 194443354 From thread  - 10
Adding 792426557 From thread  - 11
Adding 230130793 From thread  - 14
Output :
792426557
230130793

Next, time I will try and write about some other things on Flirting with Tasks.

MongoDB – Replication

Replication

Replication helps us achieve availability and fault-tolerance. A replica set is a set of mongo nodes that replicate data amongst each other asynchronously. One of the replica sets is primary while the rest of them will be secondary.
Writes only happen to the primary. If the primary goes down then an election happens and the new primary comes up.
Minimum number of nodes will be 3, since the election requires a majority of the original set.
If there were only 2 sets then the remaining one is not a majority and you would not be able to write.

Replica Set Elections

Regular Node : It has the data and can become primary or secondary.
Arbiter : It is just there for voting purposes. We need it if we want an even number of nodes.
It has no data on it.
Delayed/Regular : It can be set to a few hours after the nodes. It cannot become primary. It’s priority is set to 0.
Hidden Node : It cannot become a primary node. It’s priority is set to 0.

By default the reads and writes go to the primary. You can go to secondary for reading. This means that you might read stale data. The lag between nodes is not guaranteed since the process is async. If you read from secondary then what we have is “eventual consistency”.

rs.slaveOk() -- ok to read from the secondary
rs.isMaster() -- checks whether the node is master
rs.status() -- gives the current status of the replica set.

In the database locals , the collection oplogs.rs has all the operational logs. Replication happens when secondary nodes query the primary for the changes from a given timestamp. OpLog is the statement based replication log.
Replication happens in a statement driven manner?
e.g If a statement deletes 100 documents on the primary then there will 100 statements that are sent to the secondary to execute. There is no binary replication. This allows us to run different version of mongodb on different machines.

  • Try to keep the OpLog small on 64 bit machine since it defaults to a large value on 64 bit systems.
  • For replica sets don’t use localhost or the ip address of the machine.
  • Use a logical name, that is the best practice.
  • Use DNS.
  • Pick appropriate TTL as well.

Python Development in Visual Studio

What ..you must be mad !!

IronPython has been around for some time now. Python tools for Visual Studio have recently been given a facelift. It has been lifted to become a more complete tool which we can use for Python development with Visual Studio.
I have recently been flirting with Python since it is easier and quicker to just get some stuff done in Python. It can easily be a second language for a C# developer and I do intend to keep it in my toolbox. There are alternatives available including but not limited to PyDev plugin for eclipse and Aptana Studio which comes with all the bells bundled together.

OK..So how do we get started

If you don’t have python then I suggest you grab python 3 and above. To save yourself hours and days I suggest follow this post here Install Python on Windows

Go get all the needed mojo from here Python Tools For Visual Studio. Do remember to pick the correct msi as per your VS installation.
Do watch some videos if you feel like, they will give you a sense of what to expect. It is excruciatingly painful to get the python set up done if you are a newbie ( python guys are you listening ?? the defaults should be easy ).

A simple HellWorld would be nice. Once you finish the installation Go File -> New Project and wait awesomeness to pop up.


New Project - Python

Now, here comes the part where the defaults are stupid. Go to Tools -> Options -> Python Tools -> Check the Wait for input when process exits normally.

Now create a new Project HelloPython and wait for it to spin up. Right Click on the file and say Debug As Script.

Hello World
Press any key to continue . . .

Show me something more complicated

In the solution you should see Virtual Environments where you can add the virtual environments. In VS 2013, I see that managing the python interpreter has also moved there and the whole thing is now called PythonEnvironments. That is a nice touch and I think going forward it makes life easy. In VS 2012 you can go to your project properties and pick the python interpreter of your choice.
Creating virtual environments is easy. In VS 2012, First create a virtual environment and then add it to your project.
In VS 2013 this experience has come together nicely and the it looks more and more a native part of VS.


Pytools-VS2013

In VS 2013, as you can see I have pymongo installed ( using pip ). Right click on Python 64 bit … and install package to choose from pip of easy-install. In VS 2012 you can right click on your environment to achieve the same goal.

Back to some code.


from pymongo import MongoClient
import pprint

def hello():
    client = MongoClient("localhost",27017)
    database = client['blog']
    result = database.posts.find_one()
    pprint.pprint(result)

hello();

Are there any obvious benefits to run Python in VS ? I think the familiarity of the environment and the easy of use are compelling reasons. Refactoring and all the big IDE goodies are just nice add-ons.

This feels nice !!

PowerCMD – Console Awesoness

Consoles are back in vogue for some strange reason. Everyone wants cholocatey and other power tools to run on the command line. With stuff like mongodb, node etc.. running on the command line we need our windows command prompt to keep up with the times. If you thought Console2 was awesome then read on to find about PowerCMD.

  • Step 1.
    Go download it from here Powercmd
  • Step 2.
    Always run it as admin. On Windows 8 you can do it as shown below in the screenshot:
    Assigning admin privelges to PowerCMD Administrator Privileges for PowerCMD

  • Step 3. Start assigning your favourite shells to PowerCMD. Go to Tools -> Customize then set it up as below ( I take mongodb as the example)
    Customisation
  • Step 4.
    Start all the shells ( if you want to ) and watch it come alive.
    Press Ctrl + Enter to go to full screen mode.
    You can choose from several ( Zoink there are many and all are useful in some case or the other )
    Full Screen and Layout Aesomeness Full Screen and Layout Aesomeness
  • Step 5.
    Ever missed having the Environment Variables alongside your CMD. Go to View -> Watch Environment Variables.
    EnvironmentVariables EnvironmentVariables

All of this is neat and awesome. But this will cost you money as well. If you think tab completion thrown in with the above features makes it worth it, you can think of buying it. It is awesome, I hope someone decides to fix to the Windows CMD from the ground up at Microsoft.

Console 2 – Command Line awesomeness

1. What is this ?
Ever felt that the command prompt fall a little short of your expectations ? You want it to feel like the power toll and not just a tool.

2. Where do I get it ?
You can download it from here Console2

3. Help me set it up .
Refer to Scott Hansleman’s blog post Original Post.

Now, if you have successfully managed to get Console2 working note carefully what the last part in Scott’s post said
Console2 is a great little front-end for your existing shell, no matter what it is. Note that Console2 isn’t a shell itself, it’s just a face on whatever you are already using.

4. Let’s make the world a better place by adding Git Bash to the the tabs.


Title : GitBash
Icon : C:\Program Files (x86)\Git\etc\git.ico
Shell : C:\Program Files (x86)\Git\bin\sh.exe --login -i
Startup dir : c:\Users\Ashutosh ( or whatever your path is )

Console2

Console2

All looks nice, the world is a better place.

Team City – Recover Admin Password

The trials and tribulations of this world are beyond me. Sometimes, a simple gesture can save your life.
I lost(forgot, forgive my small brain it has to keep up with a day job) my TeamCity login details.
Searching online will lead you to this place Forgot Team City Password.

The solution there is mighty correct barring two things. One, your TeamCity data directory path is likely to C:\ProgramData\JetBrains\TeamCity\ on a windows box and not C:\users\foo\TeamCity. Second, you need to run CMD as admin.
So,


Step 1 Stop TeamCity service ( either through services.msc or the bat files ).

Step 2  Open CMD as ADMINISTRATOR
(don't forget the admin part you will go bonkers trying to figure out what is happening)
and go to C:\TeamCity\webapps\ROOT\WEB-INF\lib

Step 3 type -cp server.jar; hsqldb.jar ChangePassword USERNAME PASSWORD C:\ProgramData\JetBrain\TeamCity

If all is successful then you password has been reset and you can login. Here’s hoping to a few hours saved for anyone who comes around to reading this.