March 28, 2010

TFS 2010 Guidance

Have you heard of Visual Studio ALM Rangers team? This is an awesome combination of passionate people that breath and live with Team Foundation Server. Some of them are working in Visual Studio Product Group and some – in Microsoft Consulting Services. Many of the things they release is being reviewed and commented from us, Visual Studio ALM MVPs formerly known as Team System MVPs.

There are guidance documents updated for TFS 2010 released recently from this outstanding team. You wouldn’t want to miss those if you are doing any Team Foundation Server work.

Check out the list below and let me know what do you think!

Visual Studio 2010 Quick Reference Guidance

TFS 2010 Upgrade Guidance

Visual Studio TFS Branching Guide 2010

TFS 2010 Requirements Management

Visual Studio 2010 and Team Foundation Server 2010 Virtualization

TFS Integration Platform

Three more projects are still to be released soon:

Visual Studio 2010 Database Guidance

Visual Studio 2010 Testing Guidance

Visual Studio 2010 Architecture Tooling Guidance

March 26, 2010

The Value of Open Source

I want to bring your attention to a Virtual Path Provider module we’ve built in Propeople. By default EPiServer stores all user uploaded images in a shared folder. Since our very first EPiServer project years ago, it was clear to me that we need to store these images in database. So we have built this Virtual Path Provider that uses ASP.NET infrastructure and have been able to plug it into EPiServer. It is always nice to know that the CMS system you have implemented your solution with, has a nice and pluggable architecture to fit your needs.

We have been using this Virtual Path Provider for years now, and since we find it very valuable we decided to open it to the world.

It is important to acknowledge that our implementation is based on EPiCode DatabaseFileSystem module . Unfortunately this module is only supporting EPiServer 4 which is rather old version already.

We collaborated with EPiServer Norway in order to clean up our code and make it work even better. Because of our previous requirements, we did not implement 100% all of the features that such a module such have. EPiServer helped us to find out weak spots and made code improvements.

Some of the features that we improved are:

  • deep copy and delete of files and folders
  • removing files from caches in load balanced environment
  • running under scheduler context
  • fixed threading issues
  • dumping contents of database file system
  • copy between existing VPP and database VPP

Why do we keep images in database instead of file system? Which approach you should take? Read my next blog post with more information about pros and cons.

You will be able to find the latest release of Database Virtual Path Provider at EPiServer Extras. Feel free to use it and give feedback!

UPDATE: Vladimir Terziyski from Propeople’s EPiServer team has published a nice wrap up on how to use VirtualPathUnifiedProvider in EPiServer. You can read it here.

Deployment Dependencies

Building anything on the web more than a static HTML page, introduces a lot of technical challenges from the novice to the expert software developer. There are many reasons for that, but there is one particular field that most people get very much frustrated - Deployment.

Most software developers are unaware of architectural and operational issues that a real life production system usually has. Developers can be really good at using AJAX, jQuery, MVC or whatever you name it, but if they to deploy the solution for themselves, they will most probably fail to do that in a reasonable time.

Why is that? Is it that Deployment process is much more complicated than writing software? Or is it because Deployment should be done only by IT Operations team, not the Developers? I would say – definitely it is not! Try to give the developer the most hardest algorithm to implement or cutting edge technology to use and he will do that. And try to give the IT Op team a WCF or ASP.NET configuration file to configure for production – they will most probably fail. My point here is that deploying for real production applications requires different skill set – the one of Software Engineer.

Software Engineers are the guys that know how software applications work, they can write and debug code, most often even in production installation. They are not afraid of digging into server performance or misconfiguration problems. On the other side they know what network load balancing, web farm installation, DNS server, and static route configuration are.

The major reason Deployment is so complicated is... dependencies... and configuration of dependencies. When developer work in their company environment many of the things he is using are already preconfigured for him. He doesn’t have to think about configuring Active Directory, Mail server, Database server, user access rights or even IIS. Most of these things have been pre-configured for him by the his technical lead and he is not aware of how to do that himself.

In my next post we are going to explore one of the deployment dependencies – the file system.

March 21, 2010

How to be User Group lead

At TechEd EMEA 2009, I was invited by Tiberiu Covaci to participate in a panel discussion. We talked about what is takes to be a User Group lead, how you can be involved with community activities and what is in for you. What made the panel really interesting was to hear different opinions from people leading Dev and IT user groups through INETA and GITCA, and also how different people from Microsoft feel about user group interactions.

The sessions is recorded and available for viewing and download here:

http://www.msteched.com/online/view.aspx?tid=53efa8fa-224d-4c5f-a694-ab7ab166e8dd

Please let me know if you are interested to become user group lead. Any help is welcomed and will bring value to our community.

March 12, 2010

Data and diagnostics cannot be collected

I am setting up Microsoft Test Manager in order to playback recorded manual tests for Propeople. I created a test case and set up my environment to collect data from web server.

However when I tried to start recording of my manual test with Microsoft Test Manager, I’ve got the error below:

Data and diagnostics cannot be collected

An error occurred while initializing diagnostic data adapters.

Timed out while initializing data and diagnostics adapters.

To increase the timeout values for all diagnostic data adapters, see the following link: http://go.microsoft.com/fwlink/?LinkId=169311.

I started poking around in Event Viewer in my local machine, test controller and test agent machines and I found some errors log at the Test Controller machine (pasting full error details so next poor soul can find it).

(QTController.exe, PID 4660, Thread 26) ControllerExecution: Exception: System.Net.Sockets.SocketException (0x80004005): No such host is known

Server stack trace:
   at System.Net.Dns.GetAddrInfo(String name)
   at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6)
   at System.Net.Dns.GetHostAddresses(String hostNameOrAddress)
   at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket()
   at System.Runtime.Remoting.Channels.RemoteConnection.GetSocket()
   at System.Runtime.Remoting.Channels.SocketCache.GetSocket(String machinePortAndSid, Boolean openNew)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.SendRequestWithRetry(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
   at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.VisualStudio.TestTools.Controller.IControllerEvents.ControllerEvent(TestMessage report)
   at Microsoft.VisualStudio.TestTools.Controller.ControllerExecution.<>c__DisplayClass1d.<CallListeners>b__19(IControllerEvents eventListener)
   at Microsoft.VisualStudio.TestTools.Execution.RemoteObjectContainer`1.InvokeAsRemoteUser(Action`1 invoke)
   at Microsoft.VisualStudio.TestTools.Controller.ControllerExecution.CallListeners(TestMessage testMessage)

The “No such host is known” exception is too general and I didn’t find any more information on the web on how to find my recording problem.

I was running the manual test recording from my laptop which at that time was connected through VPN to the office. I decided that VPN could eventually give some networking restriction, so I tried running test recording from another machine that’s located inside the office. Not surprisingly though this machine produced the same error.

What come to surprise to me is that this time the error in Event Log on Test Controller machine was slightly different:

(QTController.exe, PID 1368, Thread 34) ControllerExecution: Exception: System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 192.168.xx.xx:50086

Server stack trace:
   at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
   at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket(EndPoint ipEndPoint)
   at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket()
   at System.Runtime.Remoting.Channels.RemoteConnection.GetSocket()
   at System.Runtime.Remoting.Channels.SocketCache.GetSocket(String machinePortAndSid, Boolean openNew)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.SendRequestWithRetry(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
   at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.VisualStudio.TestTools.Controller.IControllerEvents.ControllerEvent(TestMessage report)
   at Microsoft.VisualStudio.TestTools.Controller.ControllerExecution.<>c__DisplayClass1d.<CallListeners>b__19(IControllerEvents eventListener)
   at Microsoft.VisualStudio.TestTools.Execution.RemoteObjectContainer`1.InvokeAsRemoteUser(Action`1 invoke)
   at Microsoft.VisualStudio.TestTools.Controller.ControllerExecution.CallListeners(TestMessage testMessage)

The IP 192.168.xx.xx is the address of where I run MTM client and after reading the cause of this error in Troubleshooting errors in lab management, everything suddenly got crystal clear to me.

The way MTM works is contacting Test Controller machine and initialize it. Then Test Controller is trying to open a socket back to the MTM recorder. You can see that by running “netstat -nab” on the MTM machine and look for something like this:

TCP    0.0.0.0:50086          0.0.0.0:0              LISTENING
[mtm.exe]

So in order to fix all errors of this kind you must make sure that your Test Controller can connect to the machine running MTM. In my case – the second error was produced because MTM machine was blocking the port because of the firewall. The first error was produced because MTM machine is a VPN client and it doesn’t get registered and resolved in DNS. In order to workaround this, I created a record to my VPN client in hosts file at Test Controller machine.

I’ve got it all running except IntelliTrace data collection which I will blog in separate post.

March 05, 2010

Microsoft Days 2010 TFS sessions

Register for Microsoft Days 2010 and you can attend the TFS 2010 sessions below (also shameless plug :)).

Pay attention to the keynote as important announcement will be made, exclusively for Bulgarian TFS users.

User group members discounts apply. Contact me for more information.

Make sure that you drop by our Community Booth @MS Days 2010 and participate into exciting raffles :)

See you there.

Team Explorer for Eclipse

Microsoft has just announced the availability of Team Explorer 2010 codename “Eaglestone”.

Microsoft Visual Studio Team Explorer 2010 codename "Eaglestone" is the beta release of the Eclipse plug-in and cross-platform command-line client for Visual Studio 2010 Team Foundation Server.

  • Supported Operating Systems: Apple Mac OS X; HP-UX; IBM AIX; Linux; Solaris; Unix; Windows 7; Windows Vista; Windows XP Service Pack 2
  • Any of the following IDEs:
    -Eclipse 3.0 to 3.5 on Windows, Linux, Mac OS X, Solaris, AIX, or HP-UX.
    -IBM Rational Application Developer 6.0 to 7.5 on Windows.
    -IBM Rational Application Developer 7.0 to 7.5 on Linux.
    -Additional IDEs based on Eclipse 3.0 to 3.5 are supported including Adobe Flex Builder 3 and Aptana Studio 2.0.

    Any feedback is more than welcomed here.

    Read more about this release at Brian Harry blog post.

    Enjoy :)

    March 03, 2010

    Windows 32 bit RAM Limit (part 1)

    I just upgraded my Lenovo R61 laptop from 3GB to 4GB. After the upgrade Lenovo technician showed me that BIOS was saying - 4GB memory installed. When I started Windows 7, I fired up Task Manager and I can see that Total Physical Memory was 3,096MB or just 3GB. When I ask the technician what is wrong, I got the answer –“Well, Windows 32 bit can only show up to 3GB. If you need to use 4GB you need to upgrade to Windows 7 64 bit.”. My first reaction was - “But wait, 32 bit means that overall 4GB of memory can be accessed, right? So why do I only see 3GB of total memory.". Then he said something about memory bus, addressing and Windows hardware support, and final line was - “Well, I need to calculate how big exactly is 32 bit number”. At this point it was clear to me that I need to figure out the issue by myself.

    Looks like Windows 2000, XP, Vista and 7 support 4GB in their 32 bit version as written here: Memory Limits for Windows Releases.

    However displaying that information is a little bit tricky as it has been changed from Windows Vista to Windows Vista SP1.

    Windows Vista (with SP1 installed) or later will show you the total physical memory installed on the machine (Windows Vista SP1 includes reporting of Installed System Memory (RAM)) by going to Control Panel/System or My Computer/Properties window. On my machine (Windows 7) this is what I see:

    Installed memory (RAM): 4.00 GB (2.99 GB usable)
    System type: 32-bit Operating System

    Windows Vista (without SP1 installed) will show you the actual memory available to Windows due to reserved memory-mapped I/O (MMIO) as the following article explains:

    The system memory that is reported in the System Information dialog box in Windows Vista is less than you expect if 4 GB of RAM is installed

    For example, if you have a video card that has 256 MB of onboard memory, that memory must be mapped within the first 4 GB of address space. If 4 GB of system memory is already installed, part of that address space must be reserved by the graphics memory mapping. Graphics memory mapping overwrites a part of the system memory. These conditions reduce the total amount of system memory that is available to the operating system.
    The reduction in available system memory depends on the devices that are installed in the computer. However, to avoid potential driver compatibility issues, the 32-bit versions of Windows Vista limit the total available memory to 3.12 GB.

    Regardless of how Windows reports memory available, the real available memory to Windows seems to be constrained to 3.12GB on 32bit version of Windows and this is what Task Manager is actually showing.

    Read more about this constraint on Windows memory FAQ.

    There is an option to make Windows 32bit to “see” memory over 4GB that is called Physical Address Extension. However this option is more like software mapping of memory regions and has lots of issues with device drivers.

    Let’s move to our next article in this research - Pushing the Limits of Windows: Physical Memory by Mark Russinovich. In his article Mark give a nice wrap up of our problem so far:

    While 4GB is the licensed limit for 32-bit client SKUs, the effective limit is actually lower and dependent on the system's chipset and connected devices. The reason is that the physical address map includes not only RAM, but device memory as well, and x86 and x64 systems map all device memory below the 4GB address boundary to remain compatible with 32-bit operating systems that don't know how to handle addresses larger than 4GB.

    However it is interesting to see that Mark’s machine has a little more memory available:

    Installed Physical Memory (RAM) 4.00 GB
    Total Physical Memory 3.50 GB

    You may think that you should never install 32bit system with 4GB of RAM, however this comment from Anna N on Mark’s post put me into thinking:

    Yes, there's a reason to have 4GB under 32bit OS, and that is because most graphic cards have 512MB or less, so even after drivers' memory holes you have usually much more than 2GB available, and that's useful for caching (so for casual usage, too), especially under Vista. 4GB is (on current platforms) better than 3GB because of dual channel memory so 2x2GB means better memory performance than 2GB+1GB. This may change on triple-channel Nehalem, maybe 3x1GB will be better.

    Looks like even though I cannot use this 1GB I just bought, still my memory works now in dual channel mode which is faster than previous configuration.

    So the question that we are left so far with is “How I can see what is taking up my memory between 3GB and 4GB memory range and is there a way to use some parts of that area?”.

    These 0.5 GB difference of memory in my machine and Mark’s are about 12% of the total memory and it can really make the difference for my heavily used laptop.

    I will continue my research in part 2 of this post.