Archive for pjsen

| May 2, 2014
No comments

Kdiff3 as Git mergetool and –auto pitfall

In Git, when merging you can sometimes observe the behavior when a conflict has been somehow auto-magically solved without any user interaction at all and without displaying resolution tool window. I have experienced this with commonly used Kdiff3 tool set up as the default mergetool. It turns out that Git has hardcoded --auto option when invoking Kdiff3. This option instructs Kdiff3 to perform merge automatically and silently and display the window only if it is not able to figure out conflict resolutions itself. My understanding is, that it is intended solely for trivial cases and most of the time the window is displayed anyway. However, I am writing this post obviously because such “feature” once has got me into trouble — the tool fixed conflict, but should not have and, of course, did it wrong. In my opinion it is undoubtedly better to always make the decision on your own, instead of relying on some not well known logic of the tool.

The solution is to define custom tool in .gitconfig with Kdiff3 executable and custom command line parameters. Here is the configuration which I use on Windows:

[merge]
    tool = kdiff3NoAuto
    conflictstyle = diff3

[mergetool "kdiff3NoAuto"]
    cmd = C:/Program\\ Files\\ \\(x86\\)/KDiff3/kdiff3.exe --L1 \"$MERGED (Base)\" --L2 \"$MERGED (Local)\" --L3 \"$MERGED (Remote)\" -o \"$MERGED\" \"$BASE\" \"$LOCAL\" \"$REMOTE\"
Category: Quick tip, Solutions |
| March 23, 2014
No comments

Enabling the net.tcp protocol in WCF running on top of IIS — the checklist

Windows Communication Foundation is becoming sort of obsolete in favor of ASP.NET Web API, which has been advertised as primary technology for building web services. However, the latter obviously cannot serve as a full equivalent of the former. WCF still is a powerful technology for enterprise class service oriented architecture. For that purposes, the decision of switching the transport protocol from http to net.tcp sooner or later must be made clearly for performance reasons. From my experience I can tell having 100% working configuration of a service hosted inside the IIS is surprisingly hard and a developer has to face a series of quirks to bring the services back to life. Let’s sum up all the activities that can help make a service working with the net.tcp.

  1. WCF Non-HTTP Activation service must be installed in Add/Remove programs applet of the Control Panel. It is not obvious, that it is a component of the operating system itself, not of the IIS.
  2. The TCP listener processes must be running. Check netstat -a to see if there is a process listening on the port of your choice (the default is 808), check the following system services and start them if need be: Net.Tcp Listener Service and Net.Tcp Port Sharing. I have observed cases, where those services were unexpectedly shut down, e.g. after restart of the operating system.
  3. IIS management: the application must have net.tcp protocol enabled in its properties, as well as the site must have bindings for that protocol configured. If you have large number of services you can use my simplistic C# program which parses the IIS global configuration file — applicationHist.config. Link to my OneDrive
  4. If this is first try of enabling the net.tcp protocol, run the following tools to ensure the components of the .NET Framework are correctly set up: c:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_regiis.exe -i and c:\Windows\Microsoft.NET\Framework\v4.0.30319\servicemodelreg.exe -ia. Use Framework64 for 64 bit system.
  5. Make sure that you are not running on the default endpoint configuration. The default configuration can be recognized in the WSDL of the service. It contains <msf:protectionlevel>EncryptAndSign</msf:protectionlevel> code which is responsible for default authorization settings. These defaults manifest themsevles in a strange symptom of the service working in Test Client and not working in target client application. It is caused by Test Client having successfully recognized default binding configuration from WSDL whereas the target application uses your custom configuration and it is very likely these two do not match.
  6. Check for the equality of the service names in an .svc file and in the Web.config file (assuming that declarative binding is used instead of programmatically created one) in section <system.serviceModel> -> <services> -> <service>
  7. Make sure that IIS has created a virtual directory for the application. Create it from Visual Studio by pressing appropriate button in the application’s properties.
Category: .NET, ASP.NET |
| February 1, 2014
No comments

One thing cmd.exe is better at than *nix shells (with default configuration)

I know the statement might be considered controversial. I even encourage you to try to prove me wrong, because I wish I knew better solution.

In my every day work I tend to use command prompt a lot. I have both cmd.exe and bash (from Git for Windows) opened all the time. My typical environment comprises numerous directories, I mean more than one hundred. Many of them share parts of their names. The names are long, dozens of characters. I have to move between them over and over again. The problem is that it is not feasible to type longish directory name many times manually.

Now, let’s suppose we have the following (shortened, of course) subdirectories structure of a directory which we are in at the moment:

..
aa
bbaa
ccaadd
eeaaff

When I would like to change the directory to bbaa in cmd.exe I type cd *aa* <Tab> <Tab>, I get the second result of auto completion, I press <Enter> and I have moved to bbaa. If I press <tab> three times, I get ccaadd. And when I press <tab> four times, I get eeaaff. This feature is brilliant. The auto completion works with wildcards and matches not only beginnings of a name. Last, but not least, it allows to cycle through suggestions while in-line editing a command.

The most important part here is: not only beginning of a name (which is, as far as I know, the behavior of a typical Unix shell) AND also ability to have the suggestions inserted in place, not only displayed them below the command prompt.

A Unix shell also does match wildcards, but only displays matched names. It does not offer (or I am not aware of it) a way to instantaneously pass matched name to a command. It only lists relevant suggestions and a user have to then manually re-edit the command so that it has desired argument. cmd.exe is better in that it allows a user to cycle through suggestions while in-line editing command argument. Which is great when it comes to long names of which only some parts can be conveniently memorized by a human.

I propose the following function which could be appended to .bashrc.

function cdg() { ls -d */ | grep -i "$1" | awk "{printf(\"%d : %s\n\", NR, \$0)}"; read choice; if [ "$choice" == "0" ]; then : ; else cd "`ls -d */ | grep -i \"$1\" | awk \"NR==$choice\"`"; fi; }

It is a simplistic function that searches through ls results with grep, parses them with awk and finally picks one of them and calls cd.

Now we can type cdg aa and we get all possible choices:

1 : aa/
2 : bbaa/
3 : ccaadd/
4 : eeaaff/

We simply type the number and we are done being moved to the desired directory. Without the need to manually re-enter the cd command with proper argument. Obviously, in cmd.exe we get this nice auto completion for every command typed in the interpreter, and my solution only solves changing directory use case in bash.

2014.02.02 UPDATE 1: After some deeper research, it turned out that the behavior of cmd.exe can be achieved in bash as well. The following line should be included into .bashrc:

bind '"\t":menu-complete'

However, my solution still serves the purpose as it uses grep -i which makes it case insensitive and thus renders it useful.

2014.02.06 UPDATE 2: I have experienced the second reason my solution is still relevant. It is much faster than pressing <tab> and waiting for the shell to suggest names. This can be observed in an environment with number of directories greater than a few, where MinGW tooling tends to be slow in general.

Category: General programming, Quick tip, Solutions |
| January 2, 2014
5 comments

The limit of 260 characters path length and mysterious error message

If you work on small and relatively simple projects (in terms of number of components) you may not encounter this limitation. But in any non trivial ‘line of business’ application it is very likely that sooner or later you will come across this troublesome problem: Visual Studio refuses to open a project when the length of its (or any of its references) file system path is longer than 260 characters. The issue is more serious as it seems to be because of its manifestation in somewhat cryptic error message (I suppose different error messages caused by this problem may be spotted in the wild as well).

error

The error message gives absolutely no clue what the real problem is. After some research I was aware of the existence of this limitation, but I could not believe the path issue can end up with such error message. Eventually, I decided to conduct in-depth investigation with some advanced debugging tools to confirm the root cause of the problem. I followed the great advice from Case of the Unexplained series by Mark Russinovich: when in doubt, run Process Monitor. The picture below shows file system activity of devenv.exe process when it is opening solution containing suspicious projects.

pm

  • We can see the querying directory event and part of its results.
  • The results comprise file names in alphabetical order. These files should be opened by Visual Studio and thus the appropriate project references should be loaded. All of these files are expected to take part in consecutive file system events.
  • File names in green frames are both in the result list of querying directory and are then opened by the devenv.exe process. As expected.
  • File names in red frames are in the result list of querying directory, but are missing from file system activity events further (second window in the background). And this causes the problem.
  • All files that are missing have path length longer than 260 characters.
  • All files that are correctly opened by devenv.exe and loaded into the project and whose file system events are visible in Process Monitor have path length shorter than 260 characters. Obviously, the example shows only some of them, but I analyzed them all to draw the conclusion:

This proves that project references that point to dependencies with path length longer than 260 characters were not loaded and thus prevented the whole project from being loaded properly. After moving the solution to the root directory of the drive (the single letter, e.g. C, is indeed the shortest directory name possible) the problem was solved.

To wrap up:

  1. Be aware that 260 characters limit for path does exist in Windows operating system.
  2. The observable symptoms of hitting this limitation probably will not help you solve the problem.
  3. The solution in most cases could be to rename one or more directories or to make a symbolic link in the root directory of the drive.
  4. The problem described above occured under Windows 7 SP1 64-bit and Visual Studio 2012 Professional.
  5. Do not hesitate to use Process Monitor. It is incredibly powerful tool when it comes to solving wide range of operating system problems.
Category: .NET, Solutions |
| December 27, 2013
No comments

Python script executed by cron crashes when printing Unicode string

I have written a little Python script for my personal purposes and scheduled it to be run on Raspberry Pi by cron. After some polishing work I was pretty sure it worked well and was successfully tested. What I mean by to be tested is to be executed manually from the shell and to observe the expected results. So far so good.

However when the script was run by cron, it failed at the line where it prints some string containing Unicode characters. The line executes normally when running from the shell. I suspected there is some issue with the standard output of the processes run by the cron, because in such case there is no meaningful notion of standard output.

As it turns out, programs executed by cron have no terminal attached to their standard output. According to this Stack Exchange post the standard output is sent to the system’s mail system and thus delivered to a user this way. One can easily verify this by issuing tty command in the cron and redirecting the output to a file. Something similar to this is not a terminal (message translated directly from my system with Polish locale) should be observed.

The further explanation goes as follows: if there is no terminal attached to the process, the Python interpreter cannot detect the encoding of the terminal (it has nothing to do with the system’s environmental variables describing locale. They are all global and are part of the process’ environment, but they do not affect the non-terminal device being attached to standard output). You can verify this by running a Python script that tries to output terminal’s encoding: print sys.stdout.encoding. The None can be observed. So, the interpreter falls back to Ascii encoding and crashes when printing Unicode characters.

The solution in this case was to enforce the interpreter to use Unicode encoder for standard output.

UTF8Writer = codecs.getwriter('utf8')
sys.stdout = UTF8Writer(sys.stdout)

The output is discarded at all, but these lines prevent the interpreter from using default Ascii encoder which is not appropriate for printing Unicode string.

Category: General programming, Quick tip |
| November 15, 2013
No comments

Unix shell scripting — code smells

I develop and maintain a bunch of bash shell scripts for my Raspberry Pi (e.g. downloading of files from list of urls, monitoring physical gpio power off switch etc.). I have to admit I did not pay close attention to designing them perfectly, I just wanted to get the job done. However, even in such a simple cases I have experienced myself kind of technical debt when something suddenly went wrong. It is always good to follow good design principles. Here I mean avoiding unnecessary dependencies between software modules. Even in such simple programs.

These are my quick observations of what to do and what not to do when writing either shell scripts or interpeter scripts under Unix/Linux environment.

  • DO NOT USE DOT CHARACTER in script’s code. Use variables, assign them a value once and then refer to them in the code. Relying on external assumptions that script will be executed in certain directory is bad and will hurt. It is extremely likely you will forget the (in fact, unnecessary) requirements when executing script elsewhere, e.g. from cron.
  • If you absolutely have to refer to some file located in the same directory as the script itself, consider writing another script whose solely purpose is to change the directory to meet target’s expectations and then immediately execute it. Although it would be strange to write shell script to execute another shell script, this technique can be used with other interepreted languages e.g. Python or Perl scripts.
  • DO NOT USE TILDE CHARACTER in script’s code. It has not obvious expansion rules. For instance, it works at the beginning of the word, but not in the middle. That also means, it is very likely to forget the rules and have the tilde character not expanded in location where it is expected to. Use the value of HOME variable and expand it in one of traditional ways.
Category: General programming |
« Older Entries   Recent Entries »