I am not sure if it's because of a bug in Microsoft outlook 2010, but I have seen this problem a few times in the last couple of weeks with onlook 2010 on exchange server 2003 and windows 7 pro 64-bit. The problem is that the inbox keeps updating even after all the emails have been downloaded.

Workaround:
Microsoft office 2010 install a version prior to office 2010, for example 2007, 2003. setup outlook and connect it to the serve, upgrade to the new version of outlook 2010. That should solve the problem. Now I know there should be an easier way to solve this, maybe Microsoft will have a patch for this soon.
Solution2:
Make sure the machine is up to date and all Windows and office patches are installed.